-- Logs begin at Mon 2022-07-11 13:20:49 UTC, end at Mon 2022-07-11 14:30:24 UTC. -- Jul 11 13:52:08.511474 dv-cinder-848429-3-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 11 13:52:08.520080 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 11 13:52:08.522063 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: *** Starting uWSGI 2.0.18-debian (64bit) on [Mon Jul 11 13:52:08 2022] *** Jul 11 13:52:08.522063 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 11 13:52:08.522063 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 11 13:52:08.522063 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: nodename: dv-cinder-848429-3-iscsi Jul 11 13:52:08.522063 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: machine: x86_64 Jul 11 13:52:08.522063 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: clock source: unix Jul 11 13:52:08.522063 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: pcre jit disabled Jul 11 13:52:08.522063 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: detected number of CPU cores: 4 Jul 11 13:52:08.522063 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: current working directory: / Jul 11 13:52:08.522063 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: detected binary path: /usr/bin/uwsgi-core Jul 11 13:52:08.522870 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: your processes number limit is 47680 Jul 11 13:52:08.522870 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: your memory page size is 4096 bytes Jul 11 13:52:08.522870 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: detected max file descriptor number: 2048 Jul 11 13:52:08.522870 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: lock engine: pthread robust mutexes Jul 11 13:52:08.522870 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: thunder lock: enabled Jul 11 13:52:08.522870 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 11 13:52:08.522870 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 11 13:52:08.566672 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: Python main interpreter initialized at 0x56392e0919c0 Jul 11 13:52:08.566672 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: python threads support enabled Jul 11 13:52:08.566672 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: your server socket listen backlog is limited to 100 connections Jul 11 13:52:08.566672 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: your mercy for graceful operations on workers is 90 seconds Jul 11 13:52:08.567045 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: mapped 403077 bytes (393 KB) for 2 cores Jul 11 13:52:08.567123 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: *** Operational MODE: preforking *** Jul 11 13:52:08.567361 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: *** uWSGI is running in multiple interpreter mode *** Jul 11 13:52:08.567361 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: spawned uWSGI master process (pid: 80237) Jul 11 13:52:08.567562 dv-cinder-848429-3-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 11 13:52:08.571230 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: spawned uWSGI worker 1 (pid: 80247, cores: 1) Jul 11 13:52:08.571230 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: spawned uWSGI worker 2 (pid: 80248, cores: 1) Jul 11 13:52:08.571230 dv-cinder-848429-3-iscsi devstack@keystone.service[80237]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 11 13:52:14.005969 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 13:52:14.005969 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: from cryptography.utils import int_from_bytes Jul 11 13:52:14.006793 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 13:52:14.006793 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: from cryptography.utils import int_from_bytes Jul 11 13:52:14.012391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 13:52:14.012391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: from cryptography.utils import int_from_bytes Jul 11 13:52:14.012856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 11 13:52:14.012856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: from cryptography.utils import int_from_bytes Jul 11 13:52:14.055733 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 11 13:52:14.055949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 11 13:52:14.056170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] command line args: [] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 11 13:52:14.056496 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 11 13:52:14.056692 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ================================================================================ {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 11 13:52:14.056968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] admin_token = **** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.057220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] config_dir = [] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.057406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.057588 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] config_source = [] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.057766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] debug = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.057962 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.059209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.059209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.059209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.059209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.059209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] list_limit = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.059513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] log_config_append = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.059577 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.059748 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] log_dir = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.059829 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 11 13:52:14.059898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] log_file = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.059962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 11 13:52:14.060120 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.060219 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] command line args: [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 11 13:52:14.060347 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.060482 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 11 13:52:14.060616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.060768 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ================================================================================ {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 11 13:52:14.060853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.060941 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] admin_token = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.061062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.061141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.061268 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] config_dir = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.061352 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.061430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.061558 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] config_source = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.061641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.061715 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] debug = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.061833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.061909 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.062043 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.062132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.062281 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.062388 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.062520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.062639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.062764 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.062887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.063001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.063253 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.063375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.063573 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_config_append = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.063740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.063863 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.063986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] publish_errors = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.064124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.064254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_dir = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.064378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.064495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_file = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.065141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.065363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.065483 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.065608 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.065765 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.065886 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] standard_threads = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.066035 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.066155 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.066278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.066485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.066606 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.066725 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] use_journal = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.066847 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] use_json = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.066969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.067098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] use_stderr = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.067216 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.067340 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] use_syslog = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.067340 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.067565 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.067688 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.067817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.067933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.068071 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.068199 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.068317 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.068444 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.068567 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.068691 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.068813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.068932 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.069054 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.069221 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.069340 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.069458 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.069573 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.069690 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.069801 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.069918 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.070129 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.070249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.49/identity {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.070398 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.070515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] publish_errors = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.070639 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.070764 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.070881 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.071104 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.071243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.071351 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.071456 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.071561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.071659 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.071803 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.071921 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.072047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.072160 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] standard_threads = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.072287 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.072394 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.072777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.072901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.072901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.072901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] use_journal = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.073187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.073290 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] use_json = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.073399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.073507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] use_stderr = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.073629 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.073753 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] use_syslog = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.073871 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.073987 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 11 13:52:14.074102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.074102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.074399 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.075603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.075789 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.075907 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.076062 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.076349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.076477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.076634 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.076634 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.076859 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.076982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.077139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.077258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.077376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.078043 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.078171 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.078331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.078453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.078581 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.078705 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.078822 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.078946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.079077 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.079196 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.079334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.079334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.079581 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.079719 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.079848 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.079967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.080084 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.080199 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.080321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.080447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.080574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.080692 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.080692 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.080910 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.081028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.081152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.081269 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.081385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.081508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.081627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.081743 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.081868 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.081991 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.082109 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.082232 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.082395 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.082514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.082714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.082841 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.082959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.083078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.083194 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.083310 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.083435 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.083553 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.083669 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.083785 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.083902 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.084024 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.084151 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.084269 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.084385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.084507 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.084632 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.084750 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.084874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.084992 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.085091 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.085189 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.085292 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.085390 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.085495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.085599 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.085703 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.085808 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.085914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.086048 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.086161 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.086262 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.086419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.086539 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.086656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.086780 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.086906 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.087014 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.087136 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.087256 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.087256 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.087450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.087573 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.087702 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.087811 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.087934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.088053 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.088170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.088293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.088413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.088536 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.088654 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.088773 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.088890 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.089007 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.089135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.089254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.089372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.089489 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.089612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.089730 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.089855 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.089973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.090097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.090205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.090363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.090484 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.090614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.090733 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.090850 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.090967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.091084 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.091202 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.091327 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.091447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.091565 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.091682 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.091799 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.091925 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.092044 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.092170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.092288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.092405 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.092524 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.092641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.092767 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.092893 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.093010 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.093128 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.093252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.093370 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.093499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.093619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.093742 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.093861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.093978 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.094095 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.094288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.094424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] auth.external = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.094542 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.094661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.094779 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.094896 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.095031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.095161 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.095289 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.095409 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] auth.password = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.095526 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.095644 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] auth.token = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.095762 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.095879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.096005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.096123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.096241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.096359 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.096476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.096600 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.096728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.096847 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.096965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.097072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.097187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.097313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.097432 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.097550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] credential.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.097668 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.097786 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.097904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.098030 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.098149 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.098266 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.098411 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.098529 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.098646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.098798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.098928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.099052 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.099178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.099298 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.099416 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.099534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.099652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.099778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.099899 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.100018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.100135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.100135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.100363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.100492 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.100619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.100747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.100866 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.100986 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.101104 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.101230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.101349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.101457 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.101574 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] federation.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.101692 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.101811 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.101943 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.external = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.102062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.102188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.102322 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.102322 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.102548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.102680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.102801 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.102920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.103040 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.password = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.103158 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.103277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] auth.token = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.103404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.103524 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.103524 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.103746 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.103746 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.103979 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.103979 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.104221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.104221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.104445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.104564 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.104682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.104682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.104914 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.105039 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.105159 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.105159 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.105396 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.105534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.105654 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.105780 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.105900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.106019 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.106139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.106268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.106437 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.106567 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.106687 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.106805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.106995 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.107128 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.107250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.107368 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.107487 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.107605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.107731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.107860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.107980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.108099 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.108218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.108345 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.108465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.108663 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.108785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.108903 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.109030 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.109156 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.109284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.109403 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.109522 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.109641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.109759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.109948 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.110080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.110200 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.110339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.110461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.110580 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.110699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.110830 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.110950 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.111069 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.111187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.111187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.111894 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.112024 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.112143 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.112262 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.112380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.112500 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.112618 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.112744 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.112864 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.112982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.113108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.113228 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.113346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.113474 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.113594 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.113719 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.113838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.113957 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.114076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.114203 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.114405 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.114528 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.114646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.114764 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.114882 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.115013 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.115133 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.115251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.115369 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.115487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.115606 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.115733 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.115853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.115971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.116079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.116196 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.116314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.116440 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.116560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.116680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.116799 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.116917 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.117035 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.117163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.117283 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.117401 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.117519 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.117637 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.117756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.117883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.118003 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.118121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.118239 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.118405 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.118526 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.118658 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.118780 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.118899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.119017 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.119141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.119259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.119386 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.119506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.119624 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.119744 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.119862 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.119981 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.120108 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.120228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.120347 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.120465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.120584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.120702 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.120831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.120950 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.121068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.121194 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.121314 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.121432 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.121562 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.121682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.121800 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.121920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.122039 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.122165 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.122284 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.122421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.122540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.122658 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.122776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.122908 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.123028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.123148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.123266 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.123391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.123512 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.123639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.123758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.123877 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.123997 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.124116 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.124235 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.124361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.124481 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.124600 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.124718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.124838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.124982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.125117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.125238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.125357 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.125476 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.125594 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.125713 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.125840 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.125961 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.126080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.126198 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.126335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.126456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] resource.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.126591 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.126712 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.126831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.126956 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.127697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.127824 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.127954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.128075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.128196 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.128315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.128433 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.128551 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.128682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.128803 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.128922 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.129039 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.129169 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.129291 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.129418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.129538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] role.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.129657 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.129775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] role.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.129892 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.130017 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.130144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.130266 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.130405 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.130524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.130643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.130761 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.130893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.131014 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.131132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.131250 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.131370 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.131489 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.131616 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.131737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.131861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.131980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.132099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.132218 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.132347 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.132467 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.132585 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.132703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.132822 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.132941 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.133137 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.133259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.133379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.133379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.133601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.133720 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.133847 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.133967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.134086 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.134205 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.134342 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.134478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.134599 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.134718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.134835 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.134953 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.135082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.135210 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.135331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.135450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.135569 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.135688 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.135807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.135933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.136053 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.136179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.136281 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.136399 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.136518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.136646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.136766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.136885 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.137008 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.137128 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.137247 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.137374 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] role.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.137493 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.137613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] role.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.137732 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.137850 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.137969 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.138096 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.138216 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] token.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.138364 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.138497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.138619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.138737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.138737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.138970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.139088 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.139211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.139332 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.139450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.139578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.139699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.139818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.139936 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.140054 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.140173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.140300 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.140420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.140538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.140656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.140775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.140894 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.141021 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.141141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.141259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.141378 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.141496 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.141614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.141741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.141862 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.141980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.142099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.142217 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.142356 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.142490 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.connection = **** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.142642 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.142774 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.142896 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.143015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.143135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.143254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.143254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.143490 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.143610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.143729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.143848 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.143967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.144095 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.144215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.144333 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.144452 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.144571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.144689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.144816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.144938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.145057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.145176 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.145295 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.145413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.145541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.145661 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.145779 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.145897 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.146025 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.146151 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.146279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.146458 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.146578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.146697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.146815 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.146934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.147067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80247) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 11 13:52:14.147188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.147188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.147188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.147188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.147188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.147188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.147188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.147188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.147188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.148152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.148152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.148152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.148152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.148152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.148152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.connection = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.148152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.148152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.148152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.149141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.149141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.149141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.149141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.149141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.149141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.149141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.149141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.149141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.150106 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.150106 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.150106 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.150106 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.150106 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.150106 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 11 13:52:14.150106 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80248) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 11 13:52:14.172286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.172509 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.172732 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.175979 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.176213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.176440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.182182 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.185187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.209716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.209944 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.210804 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.211041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.353660 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.353909 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.402350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.402656 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.403335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.403574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.467057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.467277 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.467373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.467498 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.467613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.467696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.467802 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.467915 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.468027 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.468116 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.468248 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.468466 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.493688 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.493836 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.493912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.494006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.494130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.494216 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.494373 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.494462 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.494596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.494684 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.494812 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.494897 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.495028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.495114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.506010 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.506261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.518616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.518845 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.519006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.519110 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.519240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.519325 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.519468 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.519551 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.519680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.519766 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.519891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80247) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.519975 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.520109 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.520331 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80248) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 11 13:52:14.537518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.537728 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 11 13:52:14.538043 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.538129 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.538252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 11 13:52:14.538372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 11 13:52:14.538551 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.538635 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.538804 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 11 13:52:14.538887 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.539058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.539150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.539313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.539396 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.539576 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.539833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.550605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.550879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.551144 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.551229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.551425 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.551524 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.551691 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.551972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.559920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.560518 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.563256 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.563531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.563809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.563893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.564085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.564169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.564363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.564448 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.564667 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.564752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.564935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.565028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.565195 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.565279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.565462 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.565544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.565731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.565821 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.565993 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.566076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.566263 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.566559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.592866 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.593376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.594487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.594766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.595027 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.595307 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.600596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.600869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.601119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.601265 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.601401 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.601797 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.608530 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.608763 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.609086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.609370 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.613225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.613505 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.613783 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.613868 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.614060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.614159 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.614349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.614436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.614623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.614710 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.614890 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.614983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.615155 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.615239 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.615491 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.615761 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.633573 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.633842 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.634360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.634639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.638395 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.639347 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.642004 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.642291 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.643497 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.643788 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.647914 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.648917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.651502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.652541 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.655130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.655433 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.655703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.655992 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.656206 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.656291 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.656506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.656591 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.656793 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.657090 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.657381 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.657660 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.676009 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.676286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.676550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.676809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.677216 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.677507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.677775 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.678038 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.684946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.686231 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.686674 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.686947 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.687967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.688243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.690130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.690428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.690704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.690979 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.691306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.691502 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.691784 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.692065 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.692346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.692622 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.704452 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.704724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.705004 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.705796 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.706082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.706402 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.706681 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.706971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.707251 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.707538 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.707715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.708002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.709141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.709443 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.726220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.727864 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.729480 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.731152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.732788 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.734493 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.734590 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.736113 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.736211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.736415 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.736704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.737895 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.738175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.738489 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.744001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.745586 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.745837 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.747540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.748828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.749112 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.749497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.749779 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.750777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.751075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.751466 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.751755 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.759158 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.759442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.759718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.760948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.761242 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.761521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.767519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80247) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.768346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.768629 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.768906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.769173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.769546 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.769828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.769942 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80248) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 11 13:52:14.770104 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.770225 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.770413 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.770537 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.770685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.770819 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.770954 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.771094 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.771215 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.771477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.771608 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.771746 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80247) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.771829 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.772025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.772306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.772578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.772850 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.773124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.773393 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.773671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80248) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 11 13:52:14.809803 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56392e0919c0 pid: 80247 (default app) Jul 11 13:52:14.811968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x56392e0919c0 pid: 80248 (default app) Jul 11 13:52:14.813195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50592ba3-25ca-47e2-877a-205cdea72c58 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:14.813483 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50592ba3-25ca-47e2-877a-205cdea72c58 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:14.813759 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50592ba3-25ca-47e2-877a-205cdea72c58 None None] PATH_INFO: `/v3` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:14.817988 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1/1] 10.222.0.49 () {50 vars in 791 bytes} [Mon Jul 11 13:52:14 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:22.546115 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52a9f4-ff18-4de8-af83-38cd7bde587d None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:22.546726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52a9f4-ff18-4de8-af83-38cd7bde587d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:22.546726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf52a9f4-ff18-4de8-af83-38cd7bde587d None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:22.551065 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1/2] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:22 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:52:22.556528 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab47c57-ba9a-418b-90c9-c1c21d280335 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:22.556954 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab47c57-ba9a-418b-90c9-c1c21d280335 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:22.557084 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ab47c57-ba9a-418b-90c9-c1c21d280335 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:22.625949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: INFO dbcounter [None req-6ab47c57-ba9a-418b-90c9-c1c21d280335 None None] Registered counter for database keystone Jul 11 13:52:22.648138 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG oslo_db.sqlalchemy.engines [None req-6ab47c57-ba9a-418b-90c9-c1c21d280335 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80247) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 13:52:22.656525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writer thread running {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 13:52:22.838701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG passlib.handlers.bcrypt [None req-6ab47c57-ba9a-418b-90c9-c1c21d280335 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80247) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 11 13:52:22.839030 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG passlib.handlers.bcrypt [None req-6ab47c57-ba9a-418b-90c9-c1c21d280335 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80247) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 11 13:52:22.894313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-6ab47c57-ba9a-418b-90c9-c1c21d280335 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:22.895986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6ab47c57-ba9a-418b-90c9-c1c21d280335 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:23.027743 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 2/3] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:22 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 472 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 11 13:52:23.184403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52a9f4-ff18-4de8-af83-38cd7bde587d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:23.253228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: INFO dbcounter [None req-bf52a9f4-ff18-4de8-af83-38cd7bde587d None None] Registered counter for database keystone Jul 11 13:52:23.272276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG oslo_db.sqlalchemy.engines [None req-bf52a9f4-ff18-4de8-af83-38cd7bde587d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80248) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 11 13:52:23.278519 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writer thread running {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 11 13:52:23.396724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf52a9f4-ff18-4de8-af83-38cd7bde587d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:23.400921 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dc9ac15-6eaf-4c29-a237-2c25de866858 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:23.401580 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9ac15-6eaf-4c29-a237-2c25de866858 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:23.401861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9ac15-6eaf-4c29-a237-2c25de866858 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:23.402135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dc9ac15-6eaf-4c29-a237-2c25de866858 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:23.410732 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc9ac15-6eaf-4c29-a237-2c25de866858 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:23.461493 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: WARNING keystone.common.rbac_enforcer.enforcer [None req-0dc9ac15-6eaf-4c29-a237-2c25de866858 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 11 13:52:23.461773 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dc9ac15-6eaf-4c29-a237-2c25de866858 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:23.471938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 2/4] 10.222.0.49 () {58 vars in 1324 bytes} [Mon Jul 11 13:52:23 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 13:52:23.477878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab47c57-ba9a-418b-90c9-c1c21d280335 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:23.492951 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ab47c57-ba9a-418b-90c9-c1c21d280335 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:23.497408 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4a84a95-8536-4e04-83bb-8f2867cee468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:23.498563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a84a95-8536-4e04-83bb-8f2867cee468 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:23.498858 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a84a95-8536-4e04-83bb-8f2867cee468 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:23.499141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4a84a95-8536-4e04-83bb-8f2867cee468 admin admin] PATH_INFO: `/v3/projects/5898aef39ca74b7d8b99cb1aa700e9c2` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:23.500652 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a84a95-8536-4e04-83bb-8f2867cee468 admin admin] RBAC: Authorizing `identity:get_project(project_id=5898aef39ca74b7d8b99cb1aa700e9c2)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:23.554799 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: WARNING keystone.common.rbac_enforcer.enforcer [None req-e4a84a95-8536-4e04-83bb-8f2867cee468 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 11 13:52:23.555405 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4a84a95-8536-4e04-83bb-8f2867cee468 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:23.559985 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 3/5] 10.222.0.49 () {56 vars in 1207 bytes} [Mon Jul 11 13:52:23 2022] GET /identity/v3/projects/5898aef39ca74b7d8b99cb1aa700e9c2 => generated 344 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:25.587473 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba68b78-6bf3-4fd8-a3a5-1056aa995e75 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:25.587991 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba68b78-6bf3-4fd8-a3a5-1056aa995e75 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:25.588075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba68b78-6bf3-4fd8-a3a5-1056aa995e75 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:25.592159 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 3/6] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:25 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:25.598060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49aba51-e06b-4984-9ea9-8951f31c4b27 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:25.598473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49aba51-e06b-4984-9ea9-8951f31c4b27 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:25.598660 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49aba51-e06b-4984-9ea9-8951f31c4b27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:25.617859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-f49aba51-e06b-4984-9ea9-8951f31c4b27 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:25.619422 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f49aba51-e06b-4984-9ea9-8951f31c4b27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:25.627311 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 4/7] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:25 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 11 13:52:25.649218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba68b78-6bf3-4fd8-a3a5-1056aa995e75 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:25.661793 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba68b78-6bf3-4fd8-a3a5-1056aa995e75 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:25.666020 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dacfb712-8c30-4cf9-b8d4-0256cad9c226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:25.666757 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfb712-8c30-4cf9-b8d4-0256cad9c226 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:25.667048 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfb712-8c30-4cf9-b8d4-0256cad9c226 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:25.667328 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dacfb712-8c30-4cf9-b8d4-0256cad9c226 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:25.675163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacfb712-8c30-4cf9-b8d4-0256cad9c226 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:25.676939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dacfb712-8c30-4cf9-b8d4-0256cad9c226 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:25.686001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 4/8] 10.222.0.49 () {58 vars in 1324 bytes} [Mon Jul 11 13:52:25 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 13:52:25.692239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49aba51-e06b-4984-9ea9-8951f31c4b27 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:25.701189 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f49aba51-e06b-4984-9ea9-8951f31c4b27 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:25.706207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca9c36d-75df-4f42-92d8-c3b79471e5bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:25.707094 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9c36d-75df-4f42-92d8-c3b79471e5bb admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:25.707378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9c36d-75df-4f42-92d8-c3b79471e5bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:25.707655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca9c36d-75df-4f42-92d8-c3b79471e5bb admin admin] PATH_INFO: `/v3/users/e549d830b8bf4a6d8e16e2dad340366b` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:25.709047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9c36d-75df-4f42-92d8-c3b79471e5bb admin admin] RBAC: Authorizing `identity:get_user(user_id=e549d830b8bf4a6d8e16e2dad340366b)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:25.710921 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eca9c36d-75df-4f42-92d8-c3b79471e5bb admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:25.712484 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 5/9] 10.222.0.49 () {56 vars in 1198 bytes} [Mon Jul 11 13:52:25 2022] GET /identity/v3/users/e549d830b8bf4a6d8e16e2dad340366b => generated 247 bytes in 21 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:27.806179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa3afa-cdc6-48f3-bd9f-9232ad977ed5 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:27.806751 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa3afa-cdc6-48f3-bd9f-9232ad977ed5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:27.806751 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecfa3afa-cdc6-48f3-bd9f-9232ad977ed5 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:27.808138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 5/10] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:27.813385 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037901a-a673-48be-aa0a-edadb871c57c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:27.813823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037901a-a673-48be-aa0a-edadb871c57c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:27.813957 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4037901a-a673-48be-aa0a-edadb871c57c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:27.826262 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fee6c4-535b-4562-ae08-c30ab29fcba0 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:27.826670 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fee6c4-535b-4562-ae08-c30ab29fcba0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:27.826999 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fee6c4-535b-4562-ae08-c30ab29fcba0 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:27.828215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 6/11] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:52:27.831320 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4037901a-a673-48be-aa0a-edadb871c57c None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:27.832646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4037901a-a673-48be-aa0a-edadb871c57c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:27.834297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3364eb-8668-4c4f-8836-0f2f6cb8a39a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:27.834814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3364eb-8668-4c4f-8836-0f2f6cb8a39a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:27.834945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d3364eb-8668-4c4f-8836-0f2f6cb8a39a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:27.876533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG passlib.handlers.bcrypt [None req-7d3364eb-8668-4c4f-8836-0f2f6cb8a39a None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80248) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 11 13:52:27.876960 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG passlib.handlers.bcrypt [None req-7d3364eb-8668-4c4f-8836-0f2f6cb8a39a None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80248) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 11 13:52:27.922663 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7d3364eb-8668-4c4f-8836-0f2f6cb8a39a None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:27.924915 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7d3364eb-8668-4c4f-8836-0f2f6cb8a39a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:27.938683 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 6/12] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:27.977589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037901a-a673-48be-aa0a-edadb871c57c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:28.007467 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:28.033962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 7/13] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:52:28.066524 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3364eb-8668-4c4f-8836-0f2f6cb8a39a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:28.141583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4037901a-a673-48be-aa0a-edadb871c57c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:28.177552 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d3364eb-8668-4c4f-8836-0f2f6cb8a39a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:28.180533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:28.182293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a62d2-be21-4020-a6f3-73807c263ea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:28.183147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a62d2-be21-4020-a6f3-73807c263ea4 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:28.183432 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a62d2-be21-4020-a6f3-73807c263ea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:28.183714 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a62d2-be21-4020-a6f3-73807c263ea4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:28.196948 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b81a62d2-be21-4020-a6f3-73807c263ea4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:52:28.209106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-b81a62d2-be21-4020-a6f3-73807c263ea4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:52:28.209106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:28.209106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:28.209106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:28.209106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:28.209106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:28.209106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:28.209106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:28.209106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:28.209106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:28.209106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:28.209106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:28.209106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:52:28.209106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:52:28.209106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:52:28.209106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:52:28.210229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:52:28.210229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:52:28.210229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:52:28.210229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:52:28.210229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:28.210229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:28.210229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:28.210229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:28.210229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:28.210229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:28.210229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:28.210229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:28.210229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:28.210229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:28.210229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:28.210229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:28.212419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:28.212419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:28.212419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:28.212419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:28.212419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:52:28.212419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:52:28.212419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:52:28.212419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:52:28.212419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:52:28.212419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:52:28.212419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:52:28.212419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:52:28.212419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 7/14] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:52:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:52:28.216742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a62d2-be21-4020-a6f3-73807c263ea4 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:28.226560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121986e9-1cb7-44a4-86bc-bbbb75a11a0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:28.227499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121986e9-1cb7-44a4-86bc-bbbb75a11a0e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:28.227789 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121986e9-1cb7-44a4-86bc-bbbb75a11a0e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:28.228070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-121986e9-1cb7-44a4-86bc-bbbb75a11a0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:28.305547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121986e9-1cb7-44a4-86bc-bbbb75a11a0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:28.307149 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-121986e9-1cb7-44a4-86bc-bbbb75a11a0e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:28.330766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:28.357698 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a62d2-be21-4020-a6f3-73807c263ea4 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:28.399356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0f491-d3dd-42eb-8dff-bfa55175227d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:28.400056 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0f491-d3dd-42eb-8dff-bfa55175227d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:28.400349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0f491-d3dd-42eb-8dff-bfa55175227d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:28.400648 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06b0f491-d3dd-42eb-8dff-bfa55175227d None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:28.401456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0f491-d3dd-42eb-8dff-bfa55175227d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:28.401776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0f491-d3dd-42eb-8dff-bfa55175227d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:28.403294 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06b0f491-d3dd-42eb-8dff-bfa55175227d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:28.418250 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 8/15] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:52:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:52:28.424156 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0f491-d3dd-42eb-8dff-bfa55175227d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:28.437514 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 8/16] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:52:28 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 13:52:28.445265 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121986e9-1cb7-44a4-86bc-bbbb75a11a0e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:28.448870 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:28.539356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06b0f491-d3dd-42eb-8dff-bfa55175227d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:28.572025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-121986e9-1cb7-44a4-86bc-bbbb75a11a0e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:28.577782 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1738f-64c2-4b34-b581-9e3351f50683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:28.578774 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1738f-64c2-4b34-b581-9e3351f50683 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:28.579059 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1738f-64c2-4b34-b581-9e3351f50683 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:28.579342 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b1738f-64c2-4b34-b581-9e3351f50683 None admin] PATH_INFO: `/v3/users/e549d830b8bf4a6d8e16e2dad340366b` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:28.581296 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b1738f-64c2-4b34-b581-9e3351f50683 None admin] RBAC: Authorizing `identity:get_user(user_id=e549d830b8bf4a6d8e16e2dad340366b)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:28.582754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b1738f-64c2-4b34-b581-9e3351f50683 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:28.584394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 9/17] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:52:28 2022] GET /identity/v3/users/e549d830b8bf4a6d8e16e2dad340366b => generated 247 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:52:28.591033 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1738f-64c2-4b34-b581-9e3351f50683 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:28.605979 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:28.608710 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03c665-baa4-4ba6-bddc-61d1e1a9e218 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:28.609503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03c665-baa4-4ba6-bddc-61d1e1a9e218 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:28.609794 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03c665-baa4-4ba6-bddc-61d1e1a9e218 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:28.610079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a03c665-baa4-4ba6-bddc-61d1e1a9e218 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:28.629773 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03c665-baa4-4ba6-bddc-61d1e1a9e218 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:28.631293 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a03c665-baa4-4ba6-bddc-61d1e1a9e218 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:28.654880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-4a03c665-baa4-4ba6-bddc-61d1e1a9e218 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:52:28.654880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:28.654880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:28.654880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:28.654880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:28.654880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:28.654880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:28.654880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:28.654880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:28.654880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:28.654880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:28.654880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:28.654880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:52:28.654880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:52:28.654880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:52:28.655938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:52:28.655938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:28.655938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:28.655938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:28.655938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:28.655938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:28.655938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:28.655938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:28.655938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:28.655938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:28.655938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:28.655938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:28.655938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:28.655938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:28.655938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:28.655938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:28.655938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:28.656724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:52:28.656724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:52:28.656724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:52:28.656724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:52:28.656724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 9/18] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:52:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:52:28.662572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03c665-baa4-4ba6-bddc-61d1e1a9e218 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:28.697208 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b1738f-64c2-4b34-b581-9e3351f50683 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:28.724686 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:28.734607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a2fd2-3060-4e02-adc6-2b0a43719a85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:28.735418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a2fd2-3060-4e02-adc6-2b0a43719a85 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:28.735703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a2fd2-3060-4e02-adc6-2b0a43719a85 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:28.736017 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a63a2fd2-3060-4e02-adc6-2b0a43719a85 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:28.737247 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a2fd2-3060-4e02-adc6-2b0a43719a85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:28.738743 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a63a2fd2-3060-4e02-adc6-2b0a43719a85 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:28.740375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 10/19] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:52:28 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:52:28.746323 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a2fd2-3060-4e02-adc6-2b0a43719a85 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:28.775233 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a03c665-baa4-4ba6-bddc-61d1e1a9e218 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:28.823915 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e971a4b-37a4-4cfb-9a9e-e0fefaadd17f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:28.824983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e971a4b-37a4-4cfb-9a9e-e0fefaadd17f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:28.825388 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e971a4b-37a4-4cfb-9a9e-e0fefaadd17f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:28.826073 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e971a4b-37a4-4cfb-9a9e-e0fefaadd17f None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:28.826686 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e971a4b-37a4-4cfb-9a9e-e0fefaadd17f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:28.827060 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e971a4b-37a4-4cfb-9a9e-e0fefaadd17f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:28.829515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e971a4b-37a4-4cfb-9a9e-e0fefaadd17f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:28.856316 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:28.857690 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 10/20] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:52:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:52:28.881645 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a63a2fd2-3060-4e02-adc6-2b0a43719a85 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:28.925081 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f8efbf-584d-45f0-b916-b882c35646e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:28.926082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8efbf-584d-45f0-b916-b882c35646e3 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:28.926082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8efbf-584d-45f0-b916-b882c35646e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:28.926348 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f8efbf-584d-45f0-b916-b882c35646e3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:28.927376 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8efbf-584d-45f0-b916-b882c35646e3 None admin] RBAC: Adding query filter params (role.id=6964639a7f6c41e8a6701ef533151c41, scope.domain.id=default, user.id=e549d830b8bf4a6d8e16e2dad340366b) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:28.927552 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8efbf-584d-45f0-b916-b882c35646e3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:28.929093 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f8efbf-584d-45f0-b916-b882c35646e3 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:28.937249 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 11/21] 10.222.0.49 () {56 vars in 1322 bytes} [Mon Jul 11 13:52:28 2022] GET /identity/v3/role_assignments?user.id=e549d830b8bf4a6d8e16e2dad340366b&scope.domain.id=default&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 233 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:52:31.569786 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b5e75-348e-4cfb-9b23-10a7ed2e6376 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:31.570342 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b5e75-348e-4cfb-9b23-10a7ed2e6376 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:31.570342 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506b5e75-348e-4cfb-9b23-10a7ed2e6376 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:31.571767 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 11/22] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:52:31.588477 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c3314-3283-4725-b146-24a727426b48 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:31.588970 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c3314-3283-4725-b146-24a727426b48 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:31.589040 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289c3314-3283-4725-b146-24a727426b48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:31.611708 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-289c3314-3283-4725-b146-24a727426b48 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:31.612916 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-289c3314-3283-4725-b146-24a727426b48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:31.652504 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2306b8f-6b90-4cb2-883b-33d92db97c5e None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:31.652860 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2306b8f-6b90-4cb2-883b-33d92db97c5e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:31.653069 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2306b8f-6b90-4cb2-883b-33d92db97c5e None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:31.654870 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 12/23] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:31.659861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528a06a-b37c-4549-9e7b-d0b086dde048 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:31.660147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528a06a-b37c-4549-9e7b-d0b086dde048 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:31.660422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5528a06a-b37c-4549-9e7b-d0b086dde048 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:31.701666 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5528a06a-b37c-4549-9e7b-d0b086dde048 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:31.705555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 12/24] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:31.706704 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5528a06a-b37c-4549-9e7b-d0b086dde048 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:31.706925 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac1674-9425-4066-8a6a-c103db8695c3 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:31.707215 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac1674-9425-4066-8a6a-c103db8695c3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:31.707492 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feac1674-9425-4066-8a6a-c103db8695c3 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:31.709183 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 13/25] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:31.727013 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63d526-ec3e-4c2c-9e42-b4490630597f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:31.727600 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63d526-ec3e-4c2c-9e42-b4490630597f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:31.727886 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e63d526-ec3e-4c2c-9e42-b4490630597f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:31.754526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4e63d526-ec3e-4c2c-9e42-b4490630597f None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:31.756002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4e63d526-ec3e-4c2c-9e42-b4490630597f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:31.835239 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 13/26] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:52:31.836337 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528a06a-b37c-4549-9e7b-d0b086dde048 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:31.868388 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 14/27] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:52:31.870322 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fd6ee-0d22-4913-af4f-81b6fdb99aae None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:31.870624 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fd6ee-0d22-4913-af4f-81b6fdb99aae None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:31.871065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4fd6ee-0d22-4913-af4f-81b6fdb99aae None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:31.872945 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 15/28] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:31.873632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fd6ee-0d22-4913-af4f-81b6fdb99aae None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:31.879635 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:31.898736 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:31.962158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5528a06a-b37c-4549-9e7b-d0b086dde048 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:31.997753 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4fd6ee-0d22-4913-af4f-81b6fdb99aae None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:32.004503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb13b886-b24f-4a46-828a-3ac76b18997d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:32.005320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13b886-b24f-4a46-828a-3ac76b18997d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:32.005670 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13b886-b24f-4a46-828a-3ac76b18997d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:32.005954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb13b886-b24f-4a46-828a-3ac76b18997d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:32.020724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb13b886-b24f-4a46-828a-3ac76b18997d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:52:32.031025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-cb13b886-b24f-4a46-828a-3ac76b18997d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:52:32.031025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:32.031025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:32.031025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:32.031025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:32.031025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:32.031025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:32.031025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:32.031025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:32.031025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:32.031025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:32.031025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:32.031025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:52:32.031025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:52:32.031025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:52:32.031025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:52:32.032417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:52:32.032417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:52:32.032417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:52:32.032417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:52:32.032417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:32.032417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:32.032417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:32.032417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:32.032417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:32.032417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:32.032417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:32.032417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:32.032417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:32.032417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:32.032417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:32.032417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:32.033733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:32.033733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:32.033733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:32.033733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:32.033733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:52:32.033733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:52:32.033733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:52:32.033733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:52:32.033733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:52:32.033733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:52:32.033733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:52:32.033733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:52:32.035875 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 14/29] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:52:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:52:32.036063 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0182a14-8f29-4fca-89e6-55b336ed77a5 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:32.036267 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0182a14-8f29-4fca-89e6-55b336ed77a5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:32.036467 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0182a14-8f29-4fca-89e6-55b336ed77a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:32.043842 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88cc0d-4280-4987-a161-0c20289ef615 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:32.044544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88cc0d-4280-4987-a161-0c20289ef615 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:32.044832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88cc0d-4280-4987-a161-0c20289ef615 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:32.050001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef88cc0d-4280-4987-a161-0c20289ef615 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:32.050907 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88cc0d-4280-4987-a161-0c20289ef615 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:32.052359 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef88cc0d-4280-4987-a161-0c20289ef615 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:32.065409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d0182a14-8f29-4fca-89e6-55b336ed77a5 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:32.067185 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d0182a14-8f29-4fca-89e6-55b336ed77a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:32.152457 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 15/30] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:32.153604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0182a14-8f29-4fca-89e6-55b336ed77a5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:32.201743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:32.213393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 16/31] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Jul 11 13:52:31 2022] POST /identity/v3/roles => generated 219 bytes in 340 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:52:32.214746 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88cc0d-4280-4987-a161-0c20289ef615 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:32.290434 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0182a14-8f29-4fca-89e6-55b336ed77a5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:32.335360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e946435-4c33-4fdc-bc26-c2800ff8a2b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:32.336052 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e946435-4c33-4fdc-bc26-c2800ff8a2b0 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:32.336333 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e946435-4c33-4fdc-bc26-c2800ff8a2b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:32.336614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e946435-4c33-4fdc-bc26-c2800ff8a2b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:32.337385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e946435-4c33-4fdc-bc26-c2800ff8a2b0 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:32.342338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e946435-4c33-4fdc-bc26-c2800ff8a2b0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:32.415013 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:32.477557 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef88cc0d-4280-4987-a161-0c20289ef615 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:32.481326 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 16/32] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Jul 11 13:52:32 2022] POST /identity/v3/roles => generated 225 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:52:32.482094 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e946435-4c33-4fdc-bc26-c2800ff8a2b0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:32.566549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e9f8c-3e88-43cb-95c7-7890f7274568 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:32.568037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e9f8c-3e88-43cb-95c7-7890f7274568 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:32.568715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e9f8c-3e88-43cb-95c7-7890f7274568 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:32.569499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec5e9f8c-3e88-43cb-95c7-7890f7274568 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:32.575001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e9f8c-3e88-43cb-95c7-7890f7274568 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:32.575831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e9f8c-3e88-43cb-95c7-7890f7274568 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:32.579235 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec5e9f8c-3e88-43cb-95c7-7890f7274568 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:32.600632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 17/33] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:52:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 386 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:32.609877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e9f8c-3e88-43cb-95c7-7890f7274568 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:32.715260 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:32.726713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e946435-4c33-4fdc-bc26-c2800ff8a2b0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:32.779981 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27b95d7f-1b29-41a9-9e32-07891882a9cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:32.781362 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b95d7f-1b29-41a9-9e32-07891882a9cf None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:32.782150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b95d7f-1b29-41a9-9e32-07891882a9cf None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:32.782524 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27b95d7f-1b29-41a9-9e32-07891882a9cf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:32.784810 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec5e9f8c-3e88-43cb-95c7-7890f7274568 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:32.795216 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b95d7f-1b29-41a9-9e32-07891882a9cf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:32.796863 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27b95d7f-1b29-41a9-9e32-07891882a9cf None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:32.818914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-27b95d7f-1b29-41a9-9e32-07891882a9cf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:52:32.818914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:32.818914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:32.818914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:32.818914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:32.818914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:32.818914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:32.818914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:32.818914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:32.818914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:32.818914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:32.818914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:32.818914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:52:32.818914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:52:32.818914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:52:32.819806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:52:32.819806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:32.819806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:32.819806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:32.819806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:32.819806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:32.819806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:32.819806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:32.819806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:32.819806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:32.819806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:32.819806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:32.819806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:32.819806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:32.819806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:32.819806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:32.819806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:32.820823 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:52:32.820823 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:52:32.820823 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:52:32.820823 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:52:32.820823 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 17/34] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:52:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 339 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:52:32.822195 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb377e3d-ab78-417e-8ab1-89201ea08198 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:32.822528 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb377e3d-ab78-417e-8ab1-89201ea08198 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:32.822811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb377e3d-ab78-417e-8ab1-89201ea08198 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:32.823494 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:32.824096 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 18/35] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:32.827464 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb377e3d-ab78-417e-8ab1-89201ea08198 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:32.834061 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ba8778-1eac-4095-a705-d7a1987d6515 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:32.836556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba8778-1eac-4095-a705-d7a1987d6515 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:32.836556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba8778-1eac-4095-a705-d7a1987d6515 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:32.836556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ba8778-1eac-4095-a705-d7a1987d6515 None admin] PATH_INFO: `/v3/users/e549d830b8bf4a6d8e16e2dad340366b` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:32.837213 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba8778-1eac-4095-a705-d7a1987d6515 None admin] RBAC: Authorizing `identity:get_user(user_id=e549d830b8bf4a6d8e16e2dad340366b)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:32.839967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ba8778-1eac-4095-a705-d7a1987d6515 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:32.841620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 18/36] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:52:32 2022] GET /identity/v3/users/e549d830b8bf4a6d8e16e2dad340366b => generated 247 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:52:32.844968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d5796-c1c8-440f-a39f-c706900a7732 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:32.845304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d5796-c1c8-440f-a39f-c706900a7732 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:32.845585 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c0d5796-c1c8-440f-a39f-c706900a7732 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:32.866596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-2c0d5796-c1c8-440f-a39f-c706900a7732 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:32.868307 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2c0d5796-c1c8-440f-a39f-c706900a7732 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:32.950089 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb377e3d-ab78-417e-8ab1-89201ea08198 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:32.950446 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 19/37] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:32.950779 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d5796-c1c8-440f-a39f-c706900a7732 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:32.997822 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597a67a-fffa-43d9-98f9-d6d325615ddd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:32.998441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597a67a-fffa-43d9-98f9-d6d325615ddd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:32.998727 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597a67a-fffa-43d9-98f9-d6d325615ddd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:32.999056 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3597a67a-fffa-43d9-98f9-d6d325615ddd None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:32.999917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597a67a-fffa-43d9-98f9-d6d325615ddd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:33.000252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597a67a-fffa-43d9-98f9-d6d325615ddd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:33.001712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3597a67a-fffa-43d9-98f9-d6d325615ddd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:33.014851 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 19/38] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:52:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:33.015699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597a67a-fffa-43d9-98f9-d6d325615ddd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:33.028141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:33.075256 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c0d5796-c1c8-440f-a39f-c706900a7732 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:33.108951 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:33.121390 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdd0cf-f02c-4825-a9f2-94c77240da56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:33.122199 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdd0cf-f02c-4825-a9f2-94c77240da56 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:33.122521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdd0cf-f02c-4825-a9f2-94c77240da56 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:33.122807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdd0cf-f02c-4825-a9f2-94c77240da56 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:33.124144 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafdd0cf-f02c-4825-a9f2-94c77240da56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:33.125616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafdd0cf-f02c-4825-a9f2-94c77240da56 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:33.127552 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 20/39] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:52:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:33.132631 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdd0cf-f02c-4825-a9f2-94c77240da56 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:33.175287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3597a67a-fffa-43d9-98f9-d6d325615ddd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:33.214418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79498022-fe48-483a-8b6f-bd755ca737f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:33.215149 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79498022-fe48-483a-8b6f-bd755ca737f2 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:33.215445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79498022-fe48-483a-8b6f-bd755ca737f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:33.215717 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79498022-fe48-483a-8b6f-bd755ca737f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:33.216550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79498022-fe48-483a-8b6f-bd755ca737f2 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:33.218009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79498022-fe48-483a-8b6f-bd755ca737f2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:33.242336 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 20/40] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Jul 11 13:52:33 2022] POST /identity/v3/roles => generated 223 bytes in 219 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:52:33.242336 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79498022-fe48-483a-8b6f-bd755ca737f2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:33.343286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:33.369199 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdd0cf-f02c-4825-a9f2-94c77240da56 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:33.398908 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79498022-fe48-483a-8b6f-bd755ca737f2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:33.409708 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ef4a8db-b506-4343-8536-66d8581cd4c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:33.410444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4a8db-b506-4343-8536-66d8581cd4c1 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:33.410735 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4a8db-b506-4343-8536-66d8581cd4c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:33.411014 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ef4a8db-b506-4343-8536-66d8581cd4c1 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:33.411752 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4a8db-b506-4343-8536-66d8581cd4c1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:33.413237 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ef4a8db-b506-4343-8536-66d8581cd4c1 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:33.447806 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 21/41] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 11 13:52:33 2022] POST /identity/v3/projects => generated 301 bytes in 320 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:52:33.449484 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6801bd-8d8f-437a-9e01-8a5b3faa362f None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:33.449801 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6801bd-8d8f-437a-9e01-8a5b3faa362f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:33.450281 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6801bd-8d8f-437a-9e01-8a5b3faa362f None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:33.450971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111778c-d3f9-4d36-a431-9815ef94c87a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:33.452152 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 22/42] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:33.452708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111778c-d3f9-4d36-a431-9815ef94c87a None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:33.453209 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111778c-d3f9-4d36-a431-9815ef94c87a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:33.453721 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d111778c-d3f9-4d36-a431-9815ef94c87a None admin] PATH_INFO: `/v3/domains/default/users/e549d830b8bf4a6d8e16e2dad340366b/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:33.457386 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111778c-d3f9-4d36-a431-9815ef94c87a None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=e549d830b8bf4a6d8e16e2dad340366b, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:33.458737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b162fb-7ed7-45bd-bae3-64f19bc0c444 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:33.459037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b162fb-7ed7-45bd-bae3-64f19bc0c444 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:33.459328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9b162fb-7ed7-45bd-bae3-64f19bc0c444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:33.460098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d111778c-d3f9-4d36-a431-9815ef94c87a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:33.480522 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 21/43] 10.222.0.49 () {58 vars in 1361 bytes} [Mon Jul 11 13:52:33 2022] PUT /identity/v3/domains/default/users/e549d830b8bf4a6d8e16e2dad340366b/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:52:33.484574 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e9b162fb-7ed7-45bd-bae3-64f19bc0c444 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:33.484574 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e9b162fb-7ed7-45bd-bae3-64f19bc0c444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:33.549295 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:33.588237 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 23/44] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:52:33.612624 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111778c-d3f9-4d36-a431-9815ef94c87a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:33.763407 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d111778c-d3f9-4d36-a431-9815ef94c87a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:33.795215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:33.804645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3807385f-a3a2-4bbb-b68f-2dcfe09cd37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:33.805725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807385f-a3a2-4bbb-b68f-2dcfe09cd37d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:33.806021 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807385f-a3a2-4bbb-b68f-2dcfe09cd37d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:33.806324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3807385f-a3a2-4bbb-b68f-2dcfe09cd37d None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:33.807575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3807385f-a3a2-4bbb-b68f-2dcfe09cd37d None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:33.809068 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3807385f-a3a2-4bbb-b68f-2dcfe09cd37d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:33.810824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 22/45] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:52:33 2022] GET /identity/v3/domains/default => generated 199 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:33.817411 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b162fb-7ed7-45bd-bae3-64f19bc0c444 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:33.949897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9b162fb-7ed7-45bd-bae3-64f19bc0c444 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:33.987202 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53b98a6-7665-4a76-b7df-e8a82e707c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:33.987879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b98a6-7665-4a76-b7df-e8a82e707c54 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:33.988162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b98a6-7665-4a76-b7df-e8a82e707c54 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:33.988440 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53b98a6-7665-4a76-b7df-e8a82e707c54 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:33.989177 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b98a6-7665-4a76-b7df-e8a82e707c54 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:33.990700 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53b98a6-7665-4a76-b7df-e8a82e707c54 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:34.019847 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 24/46] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 11 13:52:33 2022] POST /identity/v3/projects => generated 312 bytes in 203 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 13:52:35.610647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ebb77-471d-4bc5-9bb3-e738a40bcc15 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:35.611653 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ebb77-471d-4bc5-9bb3-e738a40bcc15 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:35.611735 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043ebb77-471d-4bc5-9bb3-e738a40bcc15 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:35.613111 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 23/47] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:35.619341 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7f07c-875b-418d-b342-def39da54b8f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:35.619812 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7f07c-875b-418d-b342-def39da54b8f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:35.620042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28b7f07c-875b-418d-b342-def39da54b8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:35.638046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-28b7f07c-875b-418d-b342-def39da54b8f None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:35.639281 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-28b7f07c-875b-418d-b342-def39da54b8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:35.660595 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:35.710195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 25/48] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:35.733805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ebb77-471d-4bc5-9bb3-e738a40bcc15 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:35.841869 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043ebb77-471d-4bc5-9bb3-e738a40bcc15 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:35.881608 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc016d-35d3-4947-987e-349fcfb926d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:35.882358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc016d-35d3-4947-987e-349fcfb926d1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:35.882655 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc016d-35d3-4947-987e-349fcfb926d1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:35.882940 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbbc016d-35d3-4947-987e-349fcfb926d1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:35.894123 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbbc016d-35d3-4947-987e-349fcfb926d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:52:35.902072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-bbbc016d-35d3-4947-987e-349fcfb926d1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:52:35.902072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:35.902072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:35.902072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:35.902072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:35.902072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:35.902072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:35.902072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:35.902072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:35.902072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:35.902072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:35.902072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:35.902072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:52:35.902072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:52:35.902072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:52:35.902072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:52:35.902951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:52:35.902951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:52:35.902951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:52:35.902951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:52:35.902951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:35.902951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:35.902951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:35.902951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:35.902951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:35.902951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:35.902951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:35.902951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:35.902951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:35.902951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:35.902951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:35.902951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:35.903813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:35.903813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:35.903813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:35.903813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:35.903813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:52:35.903813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:52:35.903813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:52:35.903813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:52:35.903813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:52:35.903813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:52:35.903813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:52:35.903813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:52:35.903813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 24/49] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:52:35 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:52:35.911179 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7f07c-875b-418d-b342-def39da54b8f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:36.014960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28b7f07c-875b-418d-b342-def39da54b8f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:36.051518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8273ce36-67e5-44ba-b75b-adfd4e1d08a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:36.052201 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273ce36-67e5-44ba-b75b-adfd4e1d08a8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:36.052491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273ce36-67e5-44ba-b75b-adfd4e1d08a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:36.052774 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8273ce36-67e5-44ba-b75b-adfd4e1d08a8 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:36.053542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8273ce36-67e5-44ba-b75b-adfd4e1d08a8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:36.053855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8273ce36-67e5-44ba-b75b-adfd4e1d08a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:36.055547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8273ce36-67e5-44ba-b75b-adfd4e1d08a8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:36.064487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 26/50] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:52:35 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:36.070460 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc016d-35d3-4947-987e-349fcfb926d1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:36.080413 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832f953-08dc-4604-a069-630f37c2caa7 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:36.080712 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832f953-08dc-4604-a069-630f37c2caa7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:36.081003 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1832f953-08dc-4604-a069-630f37c2caa7 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:36.082497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 27/51] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:52:36.088668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b38ad5-6398-4bda-bbb9-f0137a178981 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:36.088975 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b38ad5-6398-4bda-bbb9-f0137a178981 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:36.089270 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b38ad5-6398-4bda-bbb9-f0137a178981 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:36.108531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-95b38ad5-6398-4bda-bbb9-f0137a178981 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:36.109814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-95b38ad5-6398-4bda-bbb9-f0137a178981 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:36.126250 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:36.192370 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 28/52] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:52:36.215765 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b38ad5-6398-4bda-bbb9-f0137a178981 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:36.243215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:36.297150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbbc016d-35d3-4947-987e-349fcfb926d1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:36.329330 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b38ad5-6398-4bda-bbb9-f0137a178981 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:36.335020 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e044f-f9b5-4b71-a304-8e8942bf666f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:36.335914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e044f-f9b5-4b71-a304-8e8942bf666f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:36.336199 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e044f-f9b5-4b71-a304-8e8942bf666f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:36.336477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27e044f-f9b5-4b71-a304-8e8942bf666f None admin] PATH_INFO: `/v3/users/e549d830b8bf4a6d8e16e2dad340366b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:36.337719 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e044f-f9b5-4b71-a304-8e8942bf666f None admin] RBAC: Authorizing `identity:get_user(user_id=e549d830b8bf4a6d8e16e2dad340366b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:36.339286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27e044f-f9b5-4b71-a304-8e8942bf666f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:36.340957 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 25/53] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:52:36 2022] GET /identity/v3/users/e549d830b8bf4a6d8e16e2dad340366b => generated 247 bytes in 271 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:36.346968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e044f-f9b5-4b71-a304-8e8942bf666f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:36.371502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc78a8-c14f-4f4e-8e51-60eb6fe11470 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:36.372313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc78a8-c14f-4f4e-8e51-60eb6fe11470 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:36.372603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc78a8-c14f-4f4e-8e51-60eb6fe11470 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:36.372887 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fc78a8-c14f-4f4e-8e51-60eb6fe11470 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:36.374092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc78a8-c14f-4f4e-8e51-60eb6fe11470 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:36.375590 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fc78a8-c14f-4f4e-8e51-60eb6fe11470 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:36.377227 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 29/54] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:52:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:52:36.382677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc78a8-c14f-4f4e-8e51-60eb6fe11470 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:36.400745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:36.449344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27e044f-f9b5-4b71-a304-8e8942bf666f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:36.479192 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:36.485008 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fc78a8-c14f-4f4e-8e51-60eb6fe11470 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:36.491644 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad0738-ebe4-4746-85e1-8862ad55a56c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:36.492465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad0738-ebe4-4746-85e1-8862ad55a56c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:36.492759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad0738-ebe4-4746-85e1-8862ad55a56c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:36.493044 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad0738-ebe4-4746-85e1-8862ad55a56c None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:36.494321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad0738-ebe4-4746-85e1-8862ad55a56c None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:36.495933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad0738-ebe4-4746-85e1-8862ad55a56c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:36.497668 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 26/55] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:52:36 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:52:36.503695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad0738-ebe4-4746-85e1-8862ad55a56c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:36.526232 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f3f056-743c-461f-8ff5-e363acd8fdd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:36.527398 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3f056-743c-461f-8ff5-e363acd8fdd7 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:36.527915 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3f056-743c-461f-8ff5-e363acd8fdd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:36.528398 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f3f056-743c-461f-8ff5-e363acd8fdd7 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:36.529706 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f3f056-743c-461f-8ff5-e363acd8fdd7 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:36.531760 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f3f056-743c-461f-8ff5-e363acd8fdd7 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:36.565651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 30/56] 10.222.0.49 () {60 vars in 1140 bytes} [Mon Jul 11 13:52:36 2022] POST /identity/v3/projects => generated 298 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 13:52:36.626237 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad0738-ebe4-4746-85e1-8862ad55a56c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:36.665917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd231a03-9058-4563-a1b2-5a5a565b2cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:36.666635 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd231a03-9058-4563-a1b2-5a5a565b2cfc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:36.666946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd231a03-9058-4563-a1b2-5a5a565b2cfc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:36.667232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd231a03-9058-4563-a1b2-5a5a565b2cfc None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:36.668113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd231a03-9058-4563-a1b2-5a5a565b2cfc None admin] RBAC: Adding query filter params (role.id=6964639a7f6c41e8a6701ef533151c41, scope.domain.id=default, user.id=e549d830b8bf4a6d8e16e2dad340366b) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:36.668435 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd231a03-9058-4563-a1b2-5a5a565b2cfc None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:36.669948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd231a03-9058-4563-a1b2-5a5a565b2cfc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:36.679628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 27/57] 10.222.0.49 () {56 vars in 1322 bytes} [Mon Jul 11 13:52:36 2022] GET /identity/v3/role_assignments?user.id=e549d830b8bf4a6d8e16e2dad340366b&scope.domain.id=default&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 530 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:52:38.690635 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94490ab0-c96a-41cf-bb90-6ef05155971e None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:38.691155 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94490ab0-c96a-41cf-bb90-6ef05155971e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:38.691242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94490ab0-c96a-41cf-bb90-6ef05155971e None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:38.692565 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 31/58] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:38 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:38.697782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b3134-f537-415a-8043-df32229b1b44 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:38.698190 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b3134-f537-415a-8043-df32229b1b44 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:38.698385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518b3134-f537-415a-8043-df32229b1b44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:38.715980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-518b3134-f537-415a-8043-df32229b1b44 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:38.717515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-518b3134-f537-415a-8043-df32229b1b44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:38.769115 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:38.786667 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 28/59] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:38.812013 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94490ab0-c96a-41cf-bb90-6ef05155971e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:38.881374 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:38.918467 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94490ab0-c96a-41cf-bb90-6ef05155971e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:38.953980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4560cda0-9eed-43df-b72e-d23d5967e565 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:38.954754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560cda0-9eed-43df-b72e-d23d5967e565 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:38.955047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560cda0-9eed-43df-b72e-d23d5967e565 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:38.955332 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4560cda0-9eed-43df-b72e-d23d5967e565 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:38.956551 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560cda0-9eed-43df-b72e-d23d5967e565 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:38.958019 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4560cda0-9eed-43df-b72e-d23d5967e565 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:38.959718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 32/60] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:52:38 2022] GET /identity/v3/domains/default => generated 199 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:38.965473 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b3134-f537-415a-8043-df32229b1b44 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:39.070680 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518b3134-f537-415a-8043-df32229b1b44 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:39.110405 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa0683d9-7898-4b38-8d34-5fd8975a7e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:39.111678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0683d9-7898-4b38-8d34-5fd8975a7e3e None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:39.111971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0683d9-7898-4b38-8d34-5fd8975a7e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:39.112263 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa0683d9-7898-4b38-8d34-5fd8975a7e3e None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:39.113025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0683d9-7898-4b38-8d34-5fd8975a7e3e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:39.114577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa0683d9-7898-4b38-8d34-5fd8975a7e3e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:39.141971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 29/61] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 11 13:52:38 2022] POST /identity/v3/users => generated 275 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 13:52:42.764569 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4025ef-33d1-44c3-a1e9-8efb8536ae4e None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:42.766463 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4025ef-33d1-44c3-a1e9-8efb8536ae4e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:42.767322 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc4025ef-33d1-44c3-a1e9-8efb8536ae4e None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:42.770360 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 33/62] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:42 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:42.776292 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ef41d-9d30-465f-8882-ba518f10477e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:42.777021 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ef41d-9d30-465f-8882-ba518f10477e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:42.777021 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-059ef41d-9d30-465f-8882-ba518f10477e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:42.811590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-059ef41d-9d30-465f-8882-ba518f10477e None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:42.814252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-059ef41d-9d30-465f-8882-ba518f10477e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:42.875914 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7b436-a19c-4f4f-a8a0-238785417726 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:42.877033 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7b436-a19c-4f4f-a8a0-238785417726 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:42.877915 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df7b436-a19c-4f4f-a8a0-238785417726 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:42.886335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 34/63] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:42 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:42.888170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:42.894331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b6e2e-5e62-4771-a18c-cad9110b84ac None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:42.894331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b6e2e-5e62-4771-a18c-cad9110b84ac None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:42.894331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814b6e2e-5e62-4771-a18c-cad9110b84ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:42.929424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-814b6e2e-5e62-4771-a18c-cad9110b84ac None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:42.931858 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-814b6e2e-5e62-4771-a18c-cad9110b84ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:42.977478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 30/64] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 202 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:52:43.003594 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ef41d-9d30-465f-8882-ba518f10477e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:43.096866 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 35/65] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:43.135464 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b6e2e-5e62-4771-a18c-cad9110b84ac None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:43.225590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-059ef41d-9d30-465f-8882-ba518f10477e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:43.267458 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be1fce5-3909-4db5-8c5d-b3fc5a134dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:43.268532 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1fce5-3909-4db5-8c5d-b3fc5a134dee None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:43.268999 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1fce5-3909-4db5-8c5d-b3fc5a134dee None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:43.269428 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be1fce5-3909-4db5-8c5d-b3fc5a134dee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:43.282519 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be1fce5-3909-4db5-8c5d-b3fc5a134dee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:52:43.283704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:43.290974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-0be1fce5-3909-4db5-8c5d-b3fc5a134dee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:52:43.290974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:43.290974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:43.290974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:43.290974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:43.290974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:43.290974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:43.290974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:43.290974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:43.290974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:43.290974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:43.290974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:43.290974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:52:43.290974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:52:43.290974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:52:43.290974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:52:43.291946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:52:43.291946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:52:43.291946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:52:43.291946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:52:43.291946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:43.291946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:43.291946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:43.291946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:43.291946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:43.291946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:43.291946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:43.291946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:43.291946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:43.291946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:43.291946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:43.291946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:43.292781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:43.292781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:43.292781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:43.292781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:43.292781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:52:43.292781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:52:43.292781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:52:43.292781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:52:43.292781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:52:43.292781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:52:43.292781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:52:43.292781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:52:43.295370 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 31/66] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:52:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 293 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:52:43.302369 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219536c-76a0-40ec-868e-d71ee5d030e5 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:43.302951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219536c-76a0-40ec-868e-d71ee5d030e5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:43.303411 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5219536c-76a0-40ec-868e-d71ee5d030e5 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:43.305263 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 32/67] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:43.306218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219536c-76a0-40ec-868e-d71ee5d030e5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:43.348437 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:43.378170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814b6e2e-5e62-4771-a18c-cad9110b84ac None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:43.453205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471c894b-c012-4d44-8c28-45e69241466c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:43.455064 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c894b-c012-4d44-8c28-45e69241466c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:43.456106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c894b-c012-4d44-8c28-45e69241466c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:43.456707 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471c894b-c012-4d44-8c28-45e69241466c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:43.469196 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5219536c-76a0-40ec-868e-d71ee5d030e5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:43.480014 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-471c894b-c012-4d44-8c28-45e69241466c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:52:43.502699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-471c894b-c012-4d44-8c28-45e69241466c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:52:43.502699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:43.502699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:43.502699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:43.502699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:43.502699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:43.502699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:43.502699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:43.502699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:43.502699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:43.502699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:43.502699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:43.502699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:52:43.502699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:52:43.502699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:52:43.502699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:52:43.504687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:52:43.504687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:52:43.504687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:52:43.504687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:52:43.504687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:43.504687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:43.504687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:43.504687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:43.504687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:43.504687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:43.504687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:43.504687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:43.504687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:43.504687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:43.504687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:43.504687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:43.506519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:43.506519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:43.506519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:43.506519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:43.506519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:52:43.506519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:52:43.506519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:52:43.506519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:52:43.506519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:52:43.506519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:52:43.506519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:52:43.506519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:52:43.512321 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 36/68] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:52:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 377 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:52:43.515143 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf456f1f-1b47-4252-8dc9-dd469ae9879b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:43.515885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf456f1f-1b47-4252-8dc9-dd469ae9879b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:43.516579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf456f1f-1b47-4252-8dc9-dd469ae9879b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:43.535006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b952d3-7a91-42b7-8cf8-c6f38588b59a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:43.535982 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b952d3-7a91-42b7-8cf8-c6f38588b59a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:43.536724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b952d3-7a91-42b7-8cf8-c6f38588b59a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:43.537154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b952d3-7a91-42b7-8cf8-c6f38588b59a None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:43.538094 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b952d3-7a91-42b7-8cf8-c6f38588b59a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:43.538593 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b952d3-7a91-42b7-8cf8-c6f38588b59a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:43.540170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b952d3-7a91-42b7-8cf8-c6f38588b59a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:43.540842 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-bf456f1f-1b47-4252-8dc9-dd469ae9879b None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:43.542188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bf456f1f-1b47-4252-8dc9-dd469ae9879b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:43.560097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 33/69] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:52:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:43.561298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b952d3-7a91-42b7-8cf8-c6f38588b59a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:43.657819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 37/70] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:43.659380 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf456f1f-1b47-4252-8dc9-dd469ae9879b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:43.708112 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:43.722331 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b952d3-7a91-42b7-8cf8-c6f38588b59a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:43.778652 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:43.791253 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eac1de0-0210-40c7-bad1-628faf34180e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:43.792753 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac1de0-0210-40c7-bad1-628faf34180e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:43.792753 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac1de0-0210-40c7-bad1-628faf34180e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:43.792753 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eac1de0-0210-40c7-bad1-628faf34180e None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:43.793601 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac1de0-0210-40c7-bad1-628faf34180e None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:43.793733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac1de0-0210-40c7-bad1-628faf34180e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:43.796581 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eac1de0-0210-40c7-bad1-628faf34180e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:43.810680 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 34/71] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:52:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:43.813229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542bb95-57a1-4d5a-b0b6-c3bb73e88c32 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:43.813746 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542bb95-57a1-4d5a-b0b6-c3bb73e88c32 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:43.814250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c542bb95-57a1-4d5a-b0b6-c3bb73e88c32 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:43.816471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 35/72] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:43 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:43.817533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542bb95-57a1-4d5a-b0b6-c3bb73e88c32 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:43.868116 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf456f1f-1b47-4252-8dc9-dd469ae9879b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:43.919651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a114e9-4409-4935-8ccc-3710ea88f7b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:43.920455 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a114e9-4409-4935-8ccc-3710ea88f7b2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:43.920740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a114e9-4409-4935-8ccc-3710ea88f7b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:43.921015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a114e9-4409-4935-8ccc-3710ea88f7b2 None admin] PATH_INFO: `/v3/users/e549d830b8bf4a6d8e16e2dad340366b` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:43.922349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a114e9-4409-4935-8ccc-3710ea88f7b2 None admin] RBAC: Authorizing `identity:get_user(user_id=e549d830b8bf4a6d8e16e2dad340366b)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:43.923841 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a114e9-4409-4935-8ccc-3710ea88f7b2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:43.932874 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 38/73] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:52:43 2022] GET /identity/v3/users/e549d830b8bf4a6d8e16e2dad340366b => generated 247 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:43.932874 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a114e9-4409-4935-8ccc-3710ea88f7b2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:43.987996 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c542bb95-57a1-4d5a-b0b6-c3bb73e88c32 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:44.041936 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df77b87a-a109-4264-a142-99143cccea94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:44.043314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77b87a-a109-4264-a142-99143cccea94 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:44.043420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a114e9-4409-4935-8ccc-3710ea88f7b2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:44.043786 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77b87a-a109-4264-a142-99143cccea94 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:44.044259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df77b87a-a109-4264-a142-99143cccea94 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:44.059389 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df77b87a-a109-4264-a142-99143cccea94 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:52:44.068092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-df77b87a-a109-4264-a142-99143cccea94 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:52:44.068092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:44.068092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:44.068092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:44.068092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:44.068092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:44.068092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:44.068092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:44.068092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:44.068092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:44.068092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:44.068092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:44.068092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:52:44.068092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:52:44.068092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:52:44.068092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:52:44.069529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:52:44.069529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:52:44.069529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:52:44.069529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:52:44.069529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:44.069529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:44.069529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:44.069529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:44.069529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:44.069529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:44.069529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:44.069529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:44.069529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:44.069529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:44.069529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:44.069529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:44.071009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:44.071009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:44.071009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:44.071009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:44.071009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:52:44.071009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:52:44.071009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:52:44.071009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:52:44.071009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:52:44.071009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:52:44.071009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:52:44.071009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:52:44.071009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 36/74] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:52:43 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:52:44.071009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145acd2-e074-43c9-8423-0db2a47562e5 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:44.072369 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145acd2-e074-43c9-8423-0db2a47562e5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:44.072369 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145acd2-e074-43c9-8423-0db2a47562e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:44.086716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:44.091538 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3145acd2-e074-43c9-8423-0db2a47562e5 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:44.092765 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3145acd2-e074-43c9-8423-0db2a47562e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:44.098802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790957c2-a1c6-47ef-b550-40b246a6c29e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:44.099719 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790957c2-a1c6-47ef-b550-40b246a6c29e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:44.100004 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790957c2-a1c6-47ef-b550-40b246a6c29e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:44.100283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790957c2-a1c6-47ef-b550-40b246a6c29e None admin] PATH_INFO: `/v3/users/d820d6d390fa4ca188ab16ae1466c1df` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:44.107981 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:44.118155 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790957c2-a1c6-47ef-b550-40b246a6c29e None admin] RBAC: Authorizing `identity:get_user(user_id=d820d6d390fa4ca188ab16ae1466c1df)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:44.120986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790957c2-a1c6-47ef-b550-40b246a6c29e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:44.124290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 39/75] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:52:43 2022] GET /identity/v3/users/d820d6d390fa4ca188ab16ae1466c1df => generated 275 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:44.125760 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790957c2-a1c6-47ef-b550-40b246a6c29e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:44.201135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 37/76] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:44.202751 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b512cc4-a462-4f3e-866e-4767649e56a1 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:44.203168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b512cc4-a462-4f3e-866e-4767649e56a1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:44.203643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b512cc4-a462-4f3e-866e-4767649e56a1 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:44.205627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 38/77] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:44.206502 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b512cc4-a462-4f3e-866e-4767649e56a1 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:44.288161 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790957c2-a1c6-47ef-b550-40b246a6c29e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:44.328949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eca2812-b6e5-41a6-9445-f06ef2fdf734 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:44.330183 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca2812-b6e5-41a6-9445-f06ef2fdf734 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:44.330505 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca2812-b6e5-41a6-9445-f06ef2fdf734 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:44.330796 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eca2812-b6e5-41a6-9445-f06ef2fdf734 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:44.372969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b512cc4-a462-4f3e-866e-4767649e56a1 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:44.388905 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:44.412251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca2812-b6e5-41a6-9445-f06ef2fdf734 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:44.413793 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eca2812-b6e5-41a6-9445-f06ef2fdf734 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:44.425601 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435d881-b7e8-405b-a214-fcd5bbbe2cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:44.426346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435d881-b7e8-405b-a214-fcd5bbbe2cc1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:44.426638 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435d881-b7e8-405b-a214-fcd5bbbe2cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:44.426922 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1435d881-b7e8-405b-a214-fcd5bbbe2cc1 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:44.427731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435d881-b7e8-405b-a214-fcd5bbbe2cc1 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:44.428049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435d881-b7e8-405b-a214-fcd5bbbe2cc1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:44.429517 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1435d881-b7e8-405b-a214-fcd5bbbe2cc1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:44.442804 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 39/78] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:52:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:44.444641 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435d881-b7e8-405b-a214-fcd5bbbe2cc1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:44.460081 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:44.545695 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 40/79] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:52:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:52:44.547084 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905ed39-8577-41bf-992a-5cea5f309dc2 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:44.547383 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905ed39-8577-41bf-992a-5cea5f309dc2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:44.547676 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6905ed39-8577-41bf-992a-5cea5f309dc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:44.567750 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-6905ed39-8577-41bf-992a-5cea5f309dc2 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:44.569012 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6905ed39-8577-41bf-992a-5cea5f309dc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:44.583394 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1435d881-b7e8-405b-a214-fcd5bbbe2cc1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:44.621125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17827c-d340-412c-92aa-e8fd4f2f142c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:44.621800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17827c-d340-412c-92aa-e8fd4f2f142c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:44.622085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17827c-d340-412c-92aa-e8fd4f2f142c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:44.622394 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d17827c-d340-412c-92aa-e8fd4f2f142c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:44.658248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 41/80] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:44.659412 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905ed39-8577-41bf-992a-5cea5f309dc2 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:44.675688 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:44.708811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17827c-d340-412c-92aa-e8fd4f2f142c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:44.710762 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d17827c-d340-412c-92aa-e8fd4f2f142c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:44.722217 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:44.777459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6905ed39-8577-41bf-992a-5cea5f309dc2 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:44.826450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d85722-fbb6-4d54-bbf7-a13ef3bc3e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:44.827736 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d85722-fbb6-4d54-bbf7-a13ef3bc3e87 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:44.828206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d85722-fbb6-4d54-bbf7-a13ef3bc3e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:44.828532 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d85722-fbb6-4d54-bbf7-a13ef3bc3e87 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:44.841004 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d85722-fbb6-4d54-bbf7-a13ef3bc3e87 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:52:44.842491 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 40/81] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:52:44 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:52:44.843373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17827c-d340-412c-92aa-e8fd4f2f142c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:44.854869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-07d85722-fbb6-4d54-bbf7-a13ef3bc3e87 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:52:44.854869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:44.854869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:44.854869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:44.854869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:44.854869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:44.854869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:44.854869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:44.854869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:44.854869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:44.854869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:44.854869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:44.854869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:52:44.854869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:52:44.854869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:52:44.854869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:52:44.856273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:52:44.856273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:52:44.856273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:52:44.856273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:52:44.856273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:44.856273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:44.856273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:44.856273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:44.856273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:44.856273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:44.856273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:44.856273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:44.856273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:44.856273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:44.856273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:44.856273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:44.857107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:44.857107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:44.857107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:44.857107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:44.857107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:52:44.857107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:52:44.857107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:52:44.857107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:52:44.857107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:52:44.857107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:52:44.857107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:52:44.857107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:52:44.857107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 42/82] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:52:44 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:52:44.857962 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d85722-fbb6-4d54-bbf7-a13ef3bc3e87 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:44.939848 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:44.970633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d17827c-d340-412c-92aa-e8fd4f2f142c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:44.979186 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d85722-fbb6-4d54-bbf7-a13ef3bc3e87 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:44.981071 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:45.017422 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46496c3e-2462-41de-872d-0a8771cd3ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:45.018323 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46496c3e-2462-41de-872d-0a8771cd3ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:45.018614 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46496c3e-2462-41de-872d-0a8771cd3ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:45.018896 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46496c3e-2462-41de-872d-0a8771cd3ed3 None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:45.020145 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46496c3e-2462-41de-872d-0a8771cd3ed3 None admin] RBAC: Authorizing `identity:get_project(project_id=9c32ce113f734d9cbd5e90555ab94cdd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:45.021644 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46496c3e-2462-41de-872d-0a8771cd3ed3 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:45.028430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 43/83] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:52:44 2022] GET /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:45.028430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46496c3e-2462-41de-872d-0a8771cd3ed3 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:45.028710 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ef5de-f0ce-494a-9bd3-da0bb00835ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:45.028710 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ef5de-f0ce-494a-9bd3-da0bb00835ea None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:45.028710 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ef5de-f0ce-494a-9bd3-da0bb00835ea None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:45.028710 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701ef5de-f0ce-494a-9bd3-da0bb00835ea None admin] PATH_INFO: `/v3/users/d820d6d390fa4ca188ab16ae1466c1df` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:45.030082 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ef5de-f0ce-494a-9bd3-da0bb00835ea None admin] RBAC: Authorizing `identity:get_user(user_id=d820d6d390fa4ca188ab16ae1466c1df)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:45.033515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-701ef5de-f0ce-494a-9bd3-da0bb00835ea None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:45.036567 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 41/84] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:52:44 2022] GET /identity/v3/users/d820d6d390fa4ca188ab16ae1466c1df => generated 275 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:45.038248 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ef5de-f0ce-494a-9bd3-da0bb00835ea None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:45.145859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46496c3e-2462-41de-872d-0a8771cd3ed3 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:45.169274 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-701ef5de-f0ce-494a-9bd3-da0bb00835ea None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:45.187162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9dbb33-6ef6-4a24-a0fc-7889edfa6716 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:45.187987 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9dbb33-6ef6-4a24-a0fc-7889edfa6716 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:45.188281 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9dbb33-6ef6-4a24-a0fc-7889edfa6716 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:45.188591 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9dbb33-6ef6-4a24-a0fc-7889edfa6716 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:45.189886 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9dbb33-6ef6-4a24-a0fc-7889edfa6716 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:45.191431 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf9dbb33-6ef6-4a24-a0fc-7889edfa6716 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:45.198045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 44/85] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:52:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:45.198045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9dbb33-6ef6-4a24-a0fc-7889edfa6716 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:45.209574 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:45.214294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31855ccb-e823-430a-ae53-71a2f6af96c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:45.215224 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31855ccb-e823-430a-ae53-71a2f6af96c2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:45.215512 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31855ccb-e823-430a-ae53-71a2f6af96c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:45.215791 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31855ccb-e823-430a-ae53-71a2f6af96c2 None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:45.217120 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31855ccb-e823-430a-ae53-71a2f6af96c2 None admin] RBAC: Authorizing `identity:get_project(project_id=9c32ce113f734d9cbd5e90555ab94cdd)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:45.218663 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31855ccb-e823-430a-ae53-71a2f6af96c2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:45.221079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 42/86] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:52:45 2022] GET /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:45.221833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31855ccb-e823-430a-ae53-71a2f6af96c2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:45.274512 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:45.335996 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9dbb33-6ef6-4a24-a0fc-7889edfa6716 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:45.346821 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31855ccb-e823-430a-ae53-71a2f6af96c2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:45.379795 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6aa4c-7831-4059-afe7-0cdad2a46692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:45.380489 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6aa4c-7831-4059-afe7-0cdad2a46692 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:45.380779 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6aa4c-7831-4059-afe7-0cdad2a46692 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:45.381061 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b6aa4c-7831-4059-afe7-0cdad2a46692 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:45.381853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6aa4c-7831-4059-afe7-0cdad2a46692 None admin] RBAC: Adding query filter params (name=member) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:45.382159 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6aa4c-7831-4059-afe7-0cdad2a46692 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:45.383617 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b6aa4c-7831-4059-afe7-0cdad2a46692 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:45.393786 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 45/87] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:52:45 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:45.394709 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6aa4c-7831-4059-afe7-0cdad2a46692 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:45.398674 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61087a-d1f3-4fe6-a260-cb9f6664adb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:45.399046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61087a-d1f3-4fe6-a260-cb9f6664adb4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:45.399334 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61087a-d1f3-4fe6-a260-cb9f6664adb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:45.399615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb61087a-d1f3-4fe6-a260-cb9f6664adb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:45.400487 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb61087a-d1f3-4fe6-a260-cb9f6664adb4 None admin] RBAC: Adding query filter params (role.id=6964639a7f6c41e8a6701ef533151c41, scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd, user.id=e549d830b8bf4a6d8e16e2dad340366b) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:45.400804 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb61087a-d1f3-4fe6-a260-cb9f6664adb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:45.402289 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb61087a-d1f3-4fe6-a260-cb9f6664adb4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:45.409379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 43/88] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:52:45 2022] GET /identity/v3/role_assignments?user.id=e549d830b8bf4a6d8e16e2dad340366b&scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:45.410214 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61087a-d1f3-4fe6-a260-cb9f6664adb4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:45.513107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:45.553926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb61087a-d1f3-4fe6-a260-cb9f6664adb4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:45.554140 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b6aa4c-7831-4059-afe7-0cdad2a46692 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:45.568217 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:45.605806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4543a7-c3f6-47fe-bf27-b980d43059f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:45.607232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4543a7-c3f6-47fe-bf27-b980d43059f9 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:45.607830 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4543a7-c3f6-47fe-bf27-b980d43059f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:45.608426 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4543a7-c3f6-47fe-bf27-b980d43059f9 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:45.609890 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4543a7-c3f6-47fe-bf27-b980d43059f9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:45.612741 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4543a7-c3f6-47fe-bf27-b980d43059f9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:45.618239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719ea84-22bd-421e-a9e6-4a39cc62c97c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:45.619217 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719ea84-22bd-421e-a9e6-4a39cc62c97c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:45.619656 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719ea84-22bd-421e-a9e6-4a39cc62c97c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:45.620085 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9719ea84-22bd-421e-a9e6-4a39cc62c97c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:45.644957 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 44/89] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 11 13:52:45 2022] POST /identity/v3/users => generated 289 bytes in 235 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:52:45.646237 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4543a7-c3f6-47fe-bf27-b980d43059f9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:45.754909 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719ea84-22bd-421e-a9e6-4a39cc62c97c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:45.756606 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9719ea84-22bd-421e-a9e6-4a39cc62c97c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:45.851794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:45.854131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4543a7-c3f6-47fe-bf27-b980d43059f9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:45.898337 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 46/90] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:52:45 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:52:45.898337 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719ea84-22bd-421e-a9e6-4a39cc62c97c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:45.934972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07934ea0-ab73-4735-9a5a-8a4533bb11e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:45.936126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07934ea0-ab73-4735-9a5a-8a4533bb11e1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:45.936803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07934ea0-ab73-4735-9a5a-8a4533bb11e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:45.937285 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07934ea0-ab73-4735-9a5a-8a4533bb11e1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:45.940570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07934ea0-ab73-4735-9a5a-8a4533bb11e1 None admin] RBAC: Adding query filter params (role.id=827f1f1239754bf797e64f270ba1a80c, scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd, user.id=d820d6d390fa4ca188ab16ae1466c1df) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:45.941440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07934ea0-ab73-4735-9a5a-8a4533bb11e1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:45.944410 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07934ea0-ab73-4735-9a5a-8a4533bb11e1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:45.955732 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 45/91] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:52:45 2022] GET /identity/v3/role_assignments?user.id=d820d6d390fa4ca188ab16ae1466c1df&scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd&role.id=827f1f1239754bf797e64f270ba1a80c => generated 259 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:45.957142 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07934ea0-ab73-4735-9a5a-8a4533bb11e1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:46.032593 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:46.048722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9719ea84-22bd-421e-a9e6-4a39cc62c97c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:46.122113 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57506fb9-ff2d-4fb5-92b9-3572b5db31b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:46.123277 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57506fb9-ff2d-4fb5-92b9-3572b5db31b9 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:46.123742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57506fb9-ff2d-4fb5-92b9-3572b5db31b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:46.124246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57506fb9-ff2d-4fb5-92b9-3572b5db31b9 None admin] PATH_INFO: `/v3/users/d820d6d390fa4ca188ab16ae1466c1df` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:46.126729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07934ea0-ab73-4735-9a5a-8a4533bb11e1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:46.127281 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57506fb9-ff2d-4fb5-92b9-3572b5db31b9 None admin] RBAC: Authorizing `identity:get_user(user_id=d820d6d390fa4ca188ab16ae1466c1df)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:46.130971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57506fb9-ff2d-4fb5-92b9-3572b5db31b9 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:46.133798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 47/92] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:52:45 2022] GET /identity/v3/users/d820d6d390fa4ca188ab16ae1466c1df => generated 275 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:46.143503 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57506fb9-ff2d-4fb5-92b9-3572b5db31b9 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:46.184628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5cb89-5a96-4d5f-89b3-0b635cdc5b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:46.185987 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5cb89-5a96-4d5f-89b3-0b635cdc5b0a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:46.186588 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5cb89-5a96-4d5f-89b3-0b635cdc5b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:46.187115 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda5cb89-5a96-4d5f-89b3-0b635cdc5b0a None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:46.189407 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5cb89-5a96-4d5f-89b3-0b635cdc5b0a None admin] RBAC: Authorizing `identity:get_project(project_id=9c32ce113f734d9cbd5e90555ab94cdd)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:46.191340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda5cb89-5a96-4d5f-89b3-0b635cdc5b0a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:46.193739 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 46/93] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:52:45 2022] GET /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd => generated 298 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:46.205016 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5cb89-5a96-4d5f-89b3-0b635cdc5b0a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:46.228381 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:46.310218 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57506fb9-ff2d-4fb5-92b9-3572b5db31b9 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:46.340637 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:46.346940 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda5cb89-5a96-4d5f-89b3-0b635cdc5b0a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:46.360144 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ca6db-8de5-48e7-8a54-ab83991e25fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:46.361383 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ca6db-8de5-48e7-8a54-ab83991e25fd None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:46.361684 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ca6db-8de5-48e7-8a54-ab83991e25fd None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:46.361960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c9ca6db-8de5-48e7-8a54-ab83991e25fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:46.449157 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ca6db-8de5-48e7-8a54-ab83991e25fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:46.450712 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c9ca6db-8de5-48e7-8a54-ab83991e25fd None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:46.459574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce683a-1584-4490-bb3a-22fa6ade0b4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:46.460772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce683a-1584-4490-bb3a-22fa6ade0b4c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:46.461293 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce683a-1584-4490-bb3a-22fa6ade0b4c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:46.461809 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85ce683a-1584-4490-bb3a-22fa6ade0b4c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:46.463556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce683a-1584-4490-bb3a-22fa6ade0b4c None admin] RBAC: Adding query filter params (role.id=37f000302df54045892d35f57104660a, scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd, user.id=d820d6d390fa4ca188ab16ae1466c1df) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:46.464139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce683a-1584-4490-bb3a-22fa6ade0b4c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:46.467040 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85ce683a-1584-4490-bb3a-22fa6ade0b4c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:46.497610 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 47/94] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:52:46 2022] GET /identity/v3/role_assignments?user.id=d820d6d390fa4ca188ab16ae1466c1df&scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd&role.id=37f000302df54045892d35f57104660a => generated 259 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:46.498279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:46.581662 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 48/95] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:52:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 434 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:52:46.583983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce683a-1584-4490-bb3a-22fa6ade0b4c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:46.839181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85ce683a-1584-4490-bb3a-22fa6ade0b4c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:46.909785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84181037-deaa-4538-8212-3b46a34593ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:46.911381 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84181037-deaa-4538-8212-3b46a34593ae None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:46.911894 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84181037-deaa-4538-8212-3b46a34593ae None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:46.912404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84181037-deaa-4538-8212-3b46a34593ae None admin] PATH_INFO: `/v3/projects/b29240ce139e435497a96cfb6622dcc5` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:46.914745 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84181037-deaa-4538-8212-3b46a34593ae None admin] RBAC: Authorizing `identity:get_project(project_id=b29240ce139e435497a96cfb6622dcc5)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:46.917420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84181037-deaa-4538-8212-3b46a34593ae None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:46.921022 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 48/96] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:52:46 2022] GET /identity/v3/projects/b29240ce139e435497a96cfb6622dcc5 => generated 312 bytes in 338 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:46.928446 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ca6db-8de5-48e7-8a54-ab83991e25fd None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:47.069113 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c9ca6db-8de5-48e7-8a54-ab83991e25fd None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:47.129268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed3e9ad9-12f8-4f18-931c-ac994d928015 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:47.130416 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e9ad9-12f8-4f18-931c-ac994d928015 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:47.130937 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e9ad9-12f8-4f18-931c-ac994d928015 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:47.131435 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed3e9ad9-12f8-4f18-931c-ac994d928015 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:47.133128 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e9ad9-12f8-4f18-931c-ac994d928015 None admin] RBAC: Adding query filter params (role.id=827f1f1239754bf797e64f270ba1a80c, scope.project.id=b29240ce139e435497a96cfb6622dcc5, user.id=d820d6d390fa4ca188ab16ae1466c1df) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:47.133691 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e9ad9-12f8-4f18-931c-ac994d928015 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:47.136739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed3e9ad9-12f8-4f18-931c-ac994d928015 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:47.148831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 49/97] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:52:46 2022] GET /identity/v3/role_assignments?user.id=d820d6d390fa4ca188ab16ae1466c1df&scope.project.id=b29240ce139e435497a96cfb6622dcc5&role.id=827f1f1239754bf797e64f270ba1a80c => generated 259 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:48.317333 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd6883-d51b-429a-9c7e-12787ad979a1 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:48.318433 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd6883-d51b-429a-9c7e-12787ad979a1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:48.318937 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bd6883-d51b-429a-9c7e-12787ad979a1 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:48.320634 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 49/98] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:48.326966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9b487-d0cc-49d2-8641-8d180740d9b4 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:48.328039 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9b487-d0cc-49d2-8641-8d180740d9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:48.328541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a9b487-d0cc-49d2-8641-8d180740d9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:48.334681 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:48.349418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-87a9b487-d0cc-49d2-8641-8d180740d9b4 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:48.350690 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-87a9b487-d0cc-49d2-8641-8d180740d9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:48.437575 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 50/99] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:48.460625 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd6883-d51b-429a-9c7e-12787ad979a1 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:48.503012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:48.607947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bd6883-d51b-429a-9c7e-12787ad979a1 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:48.651207 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d86ad-5db3-4c7c-9a2d-82d58f602eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:48.652038 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d86ad-5db3-4c7c-9a2d-82d58f602eeb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:48.652332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d86ad-5db3-4c7c-9a2d-82d58f602eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:48.652621 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358d86ad-5db3-4c7c-9a2d-82d58f602eeb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:48.666102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358d86ad-5db3-4c7c-9a2d-82d58f602eeb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:52:48.680612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-358d86ad-5db3-4c7c-9a2d-82d58f602eeb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:52:48.680612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:48.680612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:48.680612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:48.680612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:48.680612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:48.680612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:48.680612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:48.680612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:48.680612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:48.680612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:48.680612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:48.680612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:52:48.680612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:52:48.680612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:52:48.680612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:52:48.681521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:52:48.681521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:52:48.681521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:52:48.681521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:52:48.681521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:48.681521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:48.681521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:48.681521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:48.681521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:48.681521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:48.681521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:48.681521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:48.681521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:48.681521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:48.681521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:48.681521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:48.682245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:48.682245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:48.682245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:48.682245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:48.682245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:52:48.682245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:52:48.682245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:52:48.682245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:52:48.682245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:52:48.682245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:52:48.682245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:52:48.682245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:52:48.682245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 50/100] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:52:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:52:48.690126 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9b487-d0cc-49d2-8641-8d180740d9b4 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:48.883559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a9b487-d0cc-49d2-8641-8d180740d9b4 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:48.923658 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:48.929735 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a26b5-07fd-4bfe-b82b-4efb18106f06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:48.931283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a26b5-07fd-4bfe-b82b-4efb18106f06 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:48.932171 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a26b5-07fd-4bfe-b82b-4efb18106f06 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:48.932664 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff7a26b5-07fd-4bfe-b82b-4efb18106f06 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:48.934031 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a26b5-07fd-4bfe-b82b-4efb18106f06 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:48.934623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a26b5-07fd-4bfe-b82b-4efb18106f06 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:48.936646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff7a26b5-07fd-4bfe-b82b-4efb18106f06 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:48.947197 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 51/101] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:52:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:48.952959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d86ad-5db3-4c7c-9a2d-82d58f602eeb None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:49.059409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:49.074756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358d86ad-5db3-4c7c-9a2d-82d58f602eeb None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:49.115070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdba33-3b8e-4539-83b0-f0b9d81cbc9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:49.115990 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdba33-3b8e-4539-83b0-f0b9d81cbc9e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:49.116286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdba33-3b8e-4539-83b0-f0b9d81cbc9e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:49.116581 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dabdba33-3b8e-4539-83b0-f0b9d81cbc9e None admin] PATH_INFO: `/v3/users/e549d830b8bf4a6d8e16e2dad340366b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:49.117957 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabdba33-3b8e-4539-83b0-f0b9d81cbc9e None admin] RBAC: Authorizing `identity:get_user(user_id=e549d830b8bf4a6d8e16e2dad340366b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:49.119515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dabdba33-3b8e-4539-83b0-f0b9d81cbc9e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:49.121598 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 51/102] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:52:48 2022] GET /identity/v3/users/e549d830b8bf4a6d8e16e2dad340366b => generated 247 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:49.128779 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a26b5-07fd-4bfe-b82b-4efb18106f06 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:49.297578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff7a26b5-07fd-4bfe-b82b-4efb18106f06 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:49.359319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4c1c446-57af-4025-89a5-2510c2959f82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:49.361436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1c446-57af-4025-89a5-2510c2959f82 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:49.361962 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1c446-57af-4025-89a5-2510c2959f82 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:49.362525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4c1c446-57af-4025-89a5-2510c2959f82 None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:49.365064 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1c446-57af-4025-89a5-2510c2959f82 None admin] RBAC: Authorizing `identity:get_project(project_id=9c32ce113f734d9cbd5e90555ab94cdd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:49.368007 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4c1c446-57af-4025-89a5-2510c2959f82 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:49.371524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 52/103] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:52:49 2022] GET /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd => generated 298 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:49.381655 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdba33-3b8e-4539-83b0-f0b9d81cbc9e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:49.397894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73342877-1fdb-4023-be02-4e12c013abe1 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:49.398480 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73342877-1fdb-4023-be02-4e12c013abe1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:49.399017 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73342877-1fdb-4023-be02-4e12c013abe1 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:49.404099 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 53/104] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:49 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:49.414116 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1cce6-b6ac-49d1-bba2-7e2143388c60 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:49.415558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1cce6-b6ac-49d1-bba2-7e2143388c60 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:49.416126 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c1cce6-b6ac-49d1-bba2-7e2143388c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:49.464610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b2c1cce6-b6ac-49d1-bba2-7e2143388c60 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:49.478741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b2c1cce6-b6ac-49d1-bba2-7e2143388c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:49.586828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:49.599409 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 54/105] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:49.631632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a03e4-864c-4a6c-bce4-c58dfaf5026e None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:49.632220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a03e4-864c-4a6c-bce4-c58dfaf5026e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:49.632807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b0a03e4-864c-4a6c-bce4-c58dfaf5026e None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:49.634490 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 55/106] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:52:49.641606 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfedc950-81da-448d-a43c-f4a8559433d9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:49.642155 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfedc950-81da-448d-a43c-f4a8559433d9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:49.642620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfedc950-81da-448d-a43c-f4a8559433d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:49.663648 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dabdba33-3b8e-4539-83b0-f0b9d81cbc9e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:49.675417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-cfedc950-81da-448d-a43c-f4a8559433d9 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:49.676664 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-cfedc950-81da-448d-a43c-f4a8559433d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:49.710369 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3563fd-6893-40b2-a38a-a43207d202e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:49.711901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3563fd-6893-40b2-a38a-a43207d202e1 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:49.712695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3563fd-6893-40b2-a38a-a43207d202e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:49.713209 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd3563fd-6893-40b2-a38a-a43207d202e1 None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd/users/e549d830b8bf4a6d8e16e2dad340366b/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:49.717435 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3563fd-6893-40b2-a38a-a43207d202e1 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c32ce113f734d9cbd5e90555ab94cdd, user_id=e549d830b8bf4a6d8e16e2dad340366b, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:49.720360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd3563fd-6893-40b2-a38a-a43207d202e1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:49.741697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 52/107] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:52:49 2022] PUT /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd/users/e549d830b8bf4a6d8e16e2dad340366b/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:52:49.746507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3563fd-6893-40b2-a38a-a43207d202e1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:49.794030 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:49.819966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 56/108] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:49 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (3 switches on core 0) Jul 11 13:52:49.869269 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfedc950-81da-448d-a43c-f4a8559433d9 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:49.990022 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd3563fd-6893-40b2-a38a-a43207d202e1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:50.020136 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfedc950-81da-448d-a43c-f4a8559433d9 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:50.046657 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c114d0d-b79a-46ea-8b32-5a5c386bfca3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:50.047843 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c114d0d-b79a-46ea-8b32-5a5c386bfca3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:50.048329 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c114d0d-b79a-46ea-8b32-5a5c386bfca3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:50.048808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c114d0d-b79a-46ea-8b32-5a5c386bfca3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:50.050297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c114d0d-b79a-46ea-8b32-5a5c386bfca3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:50.052008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c114d0d-b79a-46ea-8b32-5a5c386bfca3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:50.054944 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 53/109] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:52:49 2022] GET /identity/v3/domains/default => generated 199 bytes in 311 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:50.056838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a321e8-64ef-4d39-b97e-560effdf6890 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:50.057325 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a321e8-64ef-4d39-b97e-560effdf6890 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:50.057823 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a321e8-64ef-4d39-b97e-560effdf6890 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:50.059451 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 54/110] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:50.060854 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65d87b-7075-44e9-8485-14b44d4a1f24 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:50.061338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65d87b-7075-44e9-8485-14b44d4a1f24 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:50.061800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd65d87b-7075-44e9-8485-14b44d4a1f24 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:50.063602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 55/111] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:50.065455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1377b6b-2e45-4fb1-bd71-01f4c6e674cf None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:50.065960 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1377b6b-2e45-4fb1-bd71-01f4c6e674cf None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:50.066703 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1377b6b-2e45-4fb1-bd71-01f4c6e674cf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:50.069029 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:50.084750 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff3fb5-097c-402e-a1d1-e45ecd1a31ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:50.085749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff3fb5-097c-402e-a1d1-e45ecd1a31ef None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:50.086142 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff3fb5-097c-402e-a1d1-e45ecd1a31ef None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:50.086577 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dff3fb5-097c-402e-a1d1-e45ecd1a31ef None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:50.093670 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e1377b6b-2e45-4fb1-bd71-01f4c6e674cf None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:50.095959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e1377b6b-2e45-4fb1-bd71-01f4c6e674cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:50.100952 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dff3fb5-097c-402e-a1d1-e45ecd1a31ef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:52:50.110978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-2dff3fb5-097c-402e-a1d1-e45ecd1a31ef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:52:50.110978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:50.110978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:50.110978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:50.110978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:50.110978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:50.110978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:50.110978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:50.110978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:50.110978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:50.110978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:50.110978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:50.110978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:52:50.110978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:52:50.110978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:52:50.110978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:52:50.112173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:52:50.112173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:52:50.112173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:52:50.112173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:52:50.112173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:50.112173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:50.112173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:50.112173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:50.112173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:50.112173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:50.112173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:50.112173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:50.112173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:50.112173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:50.112173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:50.112173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:50.113102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:50.113102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:50.113102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:50.113102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:50.113102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:52:50.113102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:52:50.113102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:52:50.113102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:52:50.113102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:52:50.113102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:52:50.113102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:52:50.113102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:52:50.113807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 57/112] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:52:49 2022] GET /identity/v3/roles/member => generated 84 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:52:50.119079 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff3fb5-097c-402e-a1d1-e45ecd1a31ef None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:50.193620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 56/113] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:50.195203 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3fa9e-4ceb-4e6e-b7ef-f06bdccfa1ac None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:50.195705 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3fa9e-4ceb-4e6e-b7ef-f06bdccfa1ac None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:50.196092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04b3fa9e-4ceb-4e6e-b7ef-f06bdccfa1ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:50.215521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-04b3fa9e-4ceb-4e6e-b7ef-f06bdccfa1ac None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:50.216778 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-04b3fa9e-4ceb-4e6e-b7ef-f06bdccfa1ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:50.228293 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:50.237230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dff3fb5-097c-402e-a1d1-e45ecd1a31ef None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:50.276150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f4a06-8d6d-4905-8151-e83664e11101 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:50.277186 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f4a06-8d6d-4905-8151-e83664e11101 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:50.277789 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f4a06-8d6d-4905-8151-e83664e11101 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:50.278218 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e47f4a06-8d6d-4905-8151-e83664e11101 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:50.279188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f4a06-8d6d-4905-8151-e83664e11101 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:50.280944 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e47f4a06-8d6d-4905-8151-e83664e11101 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:50.296112 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 57/114] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:50.297180 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3fa9e-4ceb-4e6e-b7ef-f06bdccfa1ac None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:50.318747 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 58/115] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 11 13:52:50 2022] POST /identity/v3/projects => generated 302 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:52:50.320882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f4a06-8d6d-4905-8151-e83664e11101 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:50.386888 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:50.474585 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e47f4a06-8d6d-4905-8151-e83664e11101 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:50.498129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04b3fa9e-4ceb-4e6e-b7ef-f06bdccfa1ac None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:50.513020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c1807-d13e-41f9-b672-943fb3084d72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:50.513899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c1807-d13e-41f9-b672-943fb3084d72 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:50.514194 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c1807-d13e-41f9-b672-943fb3084d72 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:50.514703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-506c1807-d13e-41f9-b672-943fb3084d72 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:50.528772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-506c1807-d13e-41f9-b672-943fb3084d72 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:52:50.537819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-506c1807-d13e-41f9-b672-943fb3084d72 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:52:50.537819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:50.537819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:50.537819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:50.537819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:50.537819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:50.537819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:50.537819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:50.537819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:50.537819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:50.537819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:50.537819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:50.537819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:52:50.537819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:52:50.537819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:52:50.537819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:52:50.538776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:52:50.538776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:52:50.538776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:52:50.538776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:52:50.538776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:50.538776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:50.538776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:50.538776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:50.538776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:50.538776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:50.538776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:50.538776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:50.538776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:50.538776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:50.538776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:50.538776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:50.539526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:50.539526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:50.539526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:50.539526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:50.539526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:52:50.539526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:52:50.539526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:52:50.539526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:52:50.539526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:52:50.539526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:52:50.539526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:52:50.539526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:52:50.539526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 59/116] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:52:50 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:52:50.540674 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c1807-d13e-41f9-b672-943fb3084d72 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:50.596607 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16cd10-118e-4dbd-b37d-739627b338b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:50.597528 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16cd10-118e-4dbd-b37d-739627b338b6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:50.597904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16cd10-118e-4dbd-b37d-739627b338b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:50.598291 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba16cd10-118e-4dbd-b37d-739627b338b6 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:50.599109 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16cd10-118e-4dbd-b37d-739627b338b6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:50.599427 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16cd10-118e-4dbd-b37d-739627b338b6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:50.600872 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba16cd10-118e-4dbd-b37d-739627b338b6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:50.626546 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 58/117] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:52:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:50.627281 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16cd10-118e-4dbd-b37d-739627b338b6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:50.641346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:50.667023 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-506c1807-d13e-41f9-b672-943fb3084d72 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:50.701332 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:50.705794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49fb65-e128-4bd8-a1c1-9143af494297 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:50.707012 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49fb65-e128-4bd8-a1c1-9143af494297 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:50.707450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49fb65-e128-4bd8-a1c1-9143af494297 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:50.707893 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d49fb65-e128-4bd8-a1c1-9143af494297 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:50.724980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d49fb65-e128-4bd8-a1c1-9143af494297 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:52:50.733346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-0d49fb65-e128-4bd8-a1c1-9143af494297 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 13:52:50.733346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:50.733346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:50.733346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:50.733346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:50.733346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:50.733346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:50.733346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:50.733346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:50.733346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:50.733346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:50.733346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:50.733346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:52:50.733346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:52:50.733346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:52:50.733346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:52:50.734279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:52:50.734279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:52:50.734279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:52:50.734279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:52:50.734279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:50.734279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:50.734279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:50.734279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:50.734279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:50.734279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:50.734279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:50.734279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:50.734279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:50.734279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:50.734279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:50.734279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:50.735062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:50.735062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:50.735062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:50.735062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:50.735062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:52:50.735062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:52:50.735062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:52:50.735062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:52:50.735062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:52:50.735062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:52:50.735062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 13:52:50.735062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:52:50.735062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 60/118] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:52:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:52:50.736229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49fb65-e128-4bd8-a1c1-9143af494297 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:50.758509 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba16cd10-118e-4dbd-b37d-739627b338b6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:50.800371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b11b11a-d4fe-4eba-a8ba-16045eb2ea03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:50.801403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11b11a-d4fe-4eba-a8ba-16045eb2ea03 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:50.801697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11b11a-d4fe-4eba-a8ba-16045eb2ea03 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:50.801986 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b11b11a-d4fe-4eba-a8ba-16045eb2ea03 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:50.802811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b11b11a-d4fe-4eba-a8ba-16045eb2ea03 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:50.803132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b11b11a-d4fe-4eba-a8ba-16045eb2ea03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:50.805304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b11b11a-d4fe-4eba-a8ba-16045eb2ea03 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:50.817566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 59/119] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:52:50 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:50.821542 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fd4f3-7ff5-455a-8240-4cbb4a3a0e63 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:50.821878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fd4f3-7ff5-455a-8240-4cbb4a3a0e63 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:50.822185 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1fd4f3-7ff5-455a-8240-4cbb4a3a0e63 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:50.823641 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 60/120] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:50 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:50.824499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1fd4f3-7ff5-455a-8240-4cbb4a3a0e63 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:50.870131 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d49fb65-e128-4bd8-a1c1-9143af494297 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:50.920973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92323096-e701-4a95-82da-938850aa2296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:50.921965 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92323096-e701-4a95-82da-938850aa2296 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:50.922391 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92323096-e701-4a95-82da-938850aa2296 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:50.922713 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92323096-e701-4a95-82da-938850aa2296 None admin] PATH_INFO: `/v3/users/d820d6d390fa4ca188ab16ae1466c1df` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:50.925374 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92323096-e701-4a95-82da-938850aa2296 None admin] RBAC: Authorizing `identity:get_user(user_id=d820d6d390fa4ca188ab16ae1466c1df)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:50.927557 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92323096-e701-4a95-82da-938850aa2296 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:50.930546 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 61/121] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:52:50 2022] GET /identity/v3/users/d820d6d390fa4ca188ab16ae1466c1df => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:50.931487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92323096-e701-4a95-82da-938850aa2296 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:50.942956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:50.970639 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:50.994988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1fd4f3-7ff5-455a-8240-4cbb4a3a0e63 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:51.042845 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88a722fe-f701-47c2-a1d5-c81a6c7f48a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:51.043543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a722fe-f701-47c2-a1d5-c81a6c7f48a3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:51.043839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a722fe-f701-47c2-a1d5-c81a6c7f48a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:51.044133 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88a722fe-f701-47c2-a1d5-c81a6c7f48a3 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:51.044892 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a722fe-f701-47c2-a1d5-c81a6c7f48a3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:51.045213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a722fe-f701-47c2-a1d5-c81a6c7f48a3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:51.047099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88a722fe-f701-47c2-a1d5-c81a6c7f48a3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:51.060221 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 61/122] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:52:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:51.062243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3384a9-9611-410a-82b0-3eabf4c55c03 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:51.062587 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3384a9-9611-410a-82b0-3eabf4c55c03 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:51.062878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3384a9-9611-410a-82b0-3eabf4c55c03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:51.081247 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92323096-e701-4a95-82da-938850aa2296 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:51.086015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9d3384a9-9611-410a-82b0-3eabf4c55c03 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:51.087295 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9d3384a9-9611-410a-82b0-3eabf4c55c03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:51.128117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74af5f8-42f6-4b81-b956-5c3be13ef767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:51.129672 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74af5f8-42f6-4b81-b956-5c3be13ef767 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:51.130609 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74af5f8-42f6-4b81-b956-5c3be13ef767 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:51.131371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74af5f8-42f6-4b81-b956-5c3be13ef767 None admin] PATH_INFO: `/v3/users/d820d6d390fa4ca188ab16ae1466c1df` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:51.132949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74af5f8-42f6-4b81-b956-5c3be13ef767 None admin] RBAC: Authorizing `identity:get_user(user_id=d820d6d390fa4ca188ab16ae1466c1df)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:51.134842 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74af5f8-42f6-4b81-b956-5c3be13ef767 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:51.137377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 62/123] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:52:50 2022] GET /identity/v3/users/d820d6d390fa4ca188ab16ae1466c1df => generated 275 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:51.138446 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74af5f8-42f6-4b81-b956-5c3be13ef767 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:51.195817 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 62/124] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:51.195817 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3384a9-9611-410a-82b0-3eabf4c55c03 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:51.272479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:51.299430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:51.315161 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74af5f8-42f6-4b81-b956-5c3be13ef767 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:51.339603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3384a9-9611-410a-82b0-3eabf4c55c03 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:51.389104 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f6f96-a52f-4a18-ba90-3d868e19f614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:51.390165 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f6f96-a52f-4a18-ba90-3d868e19f614 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:51.390588 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f6f96-a52f-4a18-ba90-3d868e19f614 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:51.390976 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087f6f96-a52f-4a18-ba90-3d868e19f614 None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:51.392044 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8a10a-6693-4853-a862-67bee7ed0f69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:51.392977 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f6f96-a52f-4a18-ba90-3d868e19f614 None admin] RBAC: Authorizing `identity:get_project(project_id=9c32ce113f734d9cbd5e90555ab94cdd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:51.393503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8a10a-6693-4853-a862-67bee7ed0f69 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:51.394008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8a10a-6693-4853-a862-67bee7ed0f69 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:51.394522 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67d8a10a-6693-4853-a862-67bee7ed0f69 None admin] PATH_INFO: `/v3/users/7d6a6792ad0141a1bcd2c420d03eb8fb` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:51.394992 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-087f6f96-a52f-4a18-ba90-3d868e19f614 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:51.397144 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 63/125] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:52:51 2022] GET /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd => generated 298 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:51.398479 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f6f96-a52f-4a18-ba90-3d868e19f614 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:51.424638 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d8a10a-6693-4853-a862-67bee7ed0f69 None admin] RBAC: Authorizing `identity:get_user(user_id=7d6a6792ad0141a1bcd2c420d03eb8fb)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:51.426417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67d8a10a-6693-4853-a862-67bee7ed0f69 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:51.429069 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 63/126] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:52:51 2022] GET /identity/v3/users/7d6a6792ad0141a1bcd2c420d03eb8fb => generated 289 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:51.430379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8a10a-6693-4853-a862-67bee7ed0f69 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:51.558360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67d8a10a-6693-4853-a862-67bee7ed0f69 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:51.577631 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087f6f96-a52f-4a18-ba90-3d868e19f614 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:51.583581 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:51.605270 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669c505-b4a0-4e64-a478-cb96812b33d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:51.606512 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669c505-b4a0-4e64-a478-cb96812b33d4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:51.606961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669c505-b4a0-4e64-a478-cb96812b33d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:51.607398 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0669c505-b4a0-4e64-a478-cb96812b33d4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:51.621736 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0669c505-b4a0-4e64-a478-cb96812b33d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:52:51.632102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-0669c505-b4a0-4e64-a478-cb96812b33d4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:52:51.632102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:51.632102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:51.632102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:51.632102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:51.632102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:51.632102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:51.632102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:51.632102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:51.632102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:51.632102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:51.632102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:51.632102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:52:51.632102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:52:51.632102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:52:51.632102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:52:51.633119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:52:51.633119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:52:51.633119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:52:51.633119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:52:51.633119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:51.633119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:51.633119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:51.633119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:51.633119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:51.633119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:51.633119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:51.633119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:51.633119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:51.633119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:51.633119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:51.633119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:51.633933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:51.633933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:51.633933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:51.633933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:51.633933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:52:51.633933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:52:51.633933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:52:51.633933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:52:51.633933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:52:51.633933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:52:51.633933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:52:51.633933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:52:51.642450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 64/127] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:52:51 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:52:51.642450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669c505-b4a0-4e64-a478-cb96812b33d4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:51.661814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e729b27-4f3b-4c45-a24b-2ea0d545467c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:51.663133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e729b27-4f3b-4c45-a24b-2ea0d545467c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:51.663616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e729b27-4f3b-4c45-a24b-2ea0d545467c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:51.664100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e729b27-4f3b-4c45-a24b-2ea0d545467c None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:51.666323 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e729b27-4f3b-4c45-a24b-2ea0d545467c None admin] RBAC: Authorizing `identity:get_project(project_id=9c32ce113f734d9cbd5e90555ab94cdd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:51.668772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e729b27-4f3b-4c45-a24b-2ea0d545467c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:51.671462 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 64/128] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:52:51 2022] GET /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd => generated 298 bytes in 274 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:51.672787 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e729b27-4f3b-4c45-a24b-2ea0d545467c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:51.710342 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:51.819410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e729b27-4f3b-4c45-a24b-2ea0d545467c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:51.823302 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0669c505-b4a0-4e64-a478-cb96812b33d4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:51.866252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddae56e-c447-4f78-8364-226488ed7dc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:51.867829 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddae56e-c447-4f78-8364-226488ed7dc6 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:51.868324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddae56e-c447-4f78-8364-226488ed7dc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:51.869228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddae56e-c447-4f78-8364-226488ed7dc6 None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd/users/d820d6d390fa4ca188ab16ae1466c1df/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:51.871905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddae56e-c447-4f78-8364-226488ed7dc6 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c32ce113f734d9cbd5e90555ab94cdd, user_id=d820d6d390fa4ca188ab16ae1466c1df, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:51.874379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddae56e-c447-4f78-8364-226488ed7dc6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:51.879754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b85ce3-4f19-4f5b-b60e-ccdb87b679a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:51.880530 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b85ce3-4f19-4f5b-b60e-ccdb87b679a0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:51.880895 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b85ce3-4f19-4f5b-b60e-ccdb87b679a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:51.881218 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b85ce3-4f19-4f5b-b60e-ccdb87b679a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:51.901298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 65/129] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:52:51 2022] PUT /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd/users/d820d6d390fa4ca188ab16ae1466c1df/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 264 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:52:51.902992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddae56e-c447-4f78-8364-226488ed7dc6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:51.963236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:52.047558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b85ce3-4f19-4f5b-b60e-ccdb87b679a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:52.049720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b85ce3-4f19-4f5b-b60e-ccdb87b679a0 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:52.054649 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddae56e-c447-4f78-8364-226488ed7dc6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:52.102615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27f9f2-65e0-4dcc-a544-25ec75bfc111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:52.104284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27f9f2-65e0-4dcc-a544-25ec75bfc111 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:52.104796 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27f9f2-65e0-4dcc-a544-25ec75bfc111 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:52.105306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe27f9f2-65e0-4dcc-a544-25ec75bfc111 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:52.106389 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27f9f2-65e0-4dcc-a544-25ec75bfc111 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:52.106914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27f9f2-65e0-4dcc-a544-25ec75bfc111 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:52.108673 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe27f9f2-65e0-4dcc-a544-25ec75bfc111 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:52.121579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 66/130] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:52:51 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:52.127041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:52.129293 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27f9f2-65e0-4dcc-a544-25ec75bfc111 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:52.247148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:52.251582 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe27f9f2-65e0-4dcc-a544-25ec75bfc111 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:52.290539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd42dad-16fd-4a2b-be6b-4b836cdf4293 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:52.291834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd42dad-16fd-4a2b-be6b-4b836cdf4293 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:52.292342 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd42dad-16fd-4a2b-be6b-4b836cdf4293 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:52.292812 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd42dad-16fd-4a2b-be6b-4b836cdf4293 None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd/users/d820d6d390fa4ca188ab16ae1466c1df/roles/37f000302df54045892d35f57104660a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:52.295682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd42dad-16fd-4a2b-be6b-4b836cdf4293 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c32ce113f734d9cbd5e90555ab94cdd, user_id=d820d6d390fa4ca188ab16ae1466c1df, role_id=37f000302df54045892d35f57104660a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:52.297469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd42dad-16fd-4a2b-be6b-4b836cdf4293 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:52.306884 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 65/131] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:52:51 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 634 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:52:52.308327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b85ce3-4f19-4f5b-b60e-ccdb87b679a0 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:52.319711 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 67/132] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:52:52 2022] PUT /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd/users/d820d6d390fa4ca188ab16ae1466c1df/roles/37f000302df54045892d35f57104660a => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:52:52.322386 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd42dad-16fd-4a2b-be6b-4b836cdf4293 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:52.496326 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd42dad-16fd-4a2b-be6b-4b836cdf4293 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:52.539747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6645402e-9123-4e57-8e84-f81dc0329e67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:52.540634 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645402e-9123-4e57-8e84-f81dc0329e67 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:52.540929 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645402e-9123-4e57-8e84-f81dc0329e67 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:52.541332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6645402e-9123-4e57-8e84-f81dc0329e67 None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:52.542797 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645402e-9123-4e57-8e84-f81dc0329e67 None admin] RBAC: Authorizing `identity:get_project(project_id=9c32ce113f734d9cbd5e90555ab94cdd)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:52.544342 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6645402e-9123-4e57-8e84-f81dc0329e67 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:52.546204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 68/133] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:52:52 2022] GET /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd => generated 298 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:52.547733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca03be-0c7c-43bd-a2a6-93657a579227 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:52.548037 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca03be-0c7c-43bd-a2a6-93657a579227 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:52.548327 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ca03be-0c7c-43bd-a2a6-93657a579227 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:52.550178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 69/134] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:52.555554 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca03be-0c7c-43bd-a2a6-93657a579227 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:52.563777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b85ce3-4f19-4f5b-b60e-ccdb87b679a0 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:52.590336 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:52.593736 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:52.611129 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42184bec-b3b2-4f7c-9822-a93291f0fc9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:52.612713 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42184bec-b3b2-4f7c-9822-a93291f0fc9c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:52.613003 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42184bec-b3b2-4f7c-9822-a93291f0fc9c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:52.613291 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42184bec-b3b2-4f7c-9822-a93291f0fc9c None admin] PATH_INFO: `/v3/users/d820d6d390fa4ca188ab16ae1466c1df` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:52.615104 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42184bec-b3b2-4f7c-9822-a93291f0fc9c None admin] RBAC: Authorizing `identity:get_user(user_id=d820d6d390fa4ca188ab16ae1466c1df)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:52.617164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42184bec-b3b2-4f7c-9822-a93291f0fc9c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:52.619519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 66/135] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:52:52 2022] GET /identity/v3/users/d820d6d390fa4ca188ab16ae1466c1df => generated 275 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:52.621314 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7225d-2e5d-48e9-8483-7956e0136610 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:52.621894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7225d-2e5d-48e9-8483-7956e0136610 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:52.622189 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e7225d-2e5d-48e9-8483-7956e0136610 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:52.647914 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-f7e7225d-2e5d-48e9-8483-7956e0136610 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:52.649319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f7e7225d-2e5d-48e9-8483-7956e0136610 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:52.708780 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ca03be-0c7c-43bd-a2a6-93657a579227 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:52.737753 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 67/136] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:52.738669 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7225d-2e5d-48e9-8483-7956e0136610 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:52.756075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c36e10-2e88-4b1c-85b0-39fe804eb2e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:52.756669 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c36e10-2e88-4b1c-85b0-39fe804eb2e9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:52.756961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c36e10-2e88-4b1c-85b0-39fe804eb2e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:52.757247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c36e10-2e88-4b1c-85b0-39fe804eb2e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:52.758125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c36e10-2e88-4b1c-85b0-39fe804eb2e9 None admin] RBAC: Adding query filter params (role.id=3a110b9d612545b09bf08554a2dd7eac, scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd, user.id=7d6a6792ad0141a1bcd2c420d03eb8fb) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:52.758522 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c36e10-2e88-4b1c-85b0-39fe804eb2e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:52.760140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c36e10-2e88-4b1c-85b0-39fe804eb2e9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:52.767731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 70/137] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:52:52 2022] GET /identity/v3/role_assignments?user.id=7d6a6792ad0141a1bcd2c420d03eb8fb&scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd&role.id=3a110b9d612545b09bf08554a2dd7eac => generated 259 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:52.768680 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c36e10-2e88-4b1c-85b0-39fe804eb2e9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:52.903669 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e7225d-2e5d-48e9-8483-7956e0136610 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:52.904500 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:52.917618 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c36e10-2e88-4b1c-85b0-39fe804eb2e9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:52.935763 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:52.949539 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025cee85-0ddb-4419-9183-7dc0c2ba31cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:52.950803 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cee85-0ddb-4419-9183-7dc0c2ba31cd None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:52.951265 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cee85-0ddb-4419-9183-7dc0c2ba31cd None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:52.951694 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025cee85-0ddb-4419-9183-7dc0c2ba31cd None admin] PATH_INFO: `/v3/projects/b29240ce139e435497a96cfb6622dcc5` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:52.953251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cee85-0ddb-4419-9183-7dc0c2ba31cd None admin] RBAC: Authorizing `identity:get_project(project_id=b29240ce139e435497a96cfb6622dcc5)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:52.955567 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025cee85-0ddb-4419-9183-7dc0c2ba31cd None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:52.957794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 68/138] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:52:52 2022] GET /identity/v3/projects/b29240ce139e435497a96cfb6622dcc5 => generated 312 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:52.959435 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbe99b-6603-4ac2-b041-91649e59bcf5 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:52.959725 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbe99b-6603-4ac2-b041-91649e59bcf5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:52.959999 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50bbe99b-6603-4ac2-b041-91649e59bcf5 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:52.961363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 69/139] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:52.963495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edc676f2-ceeb-40d9-87be-adf6a9d7ae58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:52.964523 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc676f2-ceeb-40d9-87be-adf6a9d7ae58 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:52.965231 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc676f2-ceeb-40d9-87be-adf6a9d7ae58 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:52.965691 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edc676f2-ceeb-40d9-87be-adf6a9d7ae58 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:52.966475 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbe99b-6603-4ac2-b041-91649e59bcf5 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:52.979916 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edc676f2-ceeb-40d9-87be-adf6a9d7ae58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:52:52.989064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-edc676f2-ceeb-40d9-87be-adf6a9d7ae58 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:52:52.989064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:52.989064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:52.989064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:52.989064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:52.989064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:52.989064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:52.989064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:52.989064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:52.989064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:52.989064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:52.989064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:52.989064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:52:52.989064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:52:52.989064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:52:52.989064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:52:52.990213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:52:52.990213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:52:52.990213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:52:52.990213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:52:52.990213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:52.990213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:52.990213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:52.990213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:52.990213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:52.990213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:52.990213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:52.990213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:52.990213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:52.990213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:52.990213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:52.990213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:52.991118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:52.991118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:52.991118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:52.991118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:52.991118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:52:52.991118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:52:52.991118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:52:52.991118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:52:52.991118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:52:52.991118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:52:52.991118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:52:52.991118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:52:52.993299 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 71/140] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:52:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:52:52.999311 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab485c7d-2133-4257-8c02-dc079c47428f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:52.999919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab485c7d-2133-4257-8c02-dc079c47428f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:53.000407 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab485c7d-2133-4257-8c02-dc079c47428f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:53.022587 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ab485c7d-2133-4257-8c02-dc079c47428f None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:53.024092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ab485c7d-2133-4257-8c02-dc079c47428f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:53.084442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50bbe99b-6603-4ac2-b041-91649e59bcf5 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:53.106075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 72/141] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:53.107374 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab485c7d-2133-4257-8c02-dc079c47428f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:53.136257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df56ff-3abd-42dc-a3d4-2c03ca95ec00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:53.137121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df56ff-3abd-42dc-a3d4-2c03ca95ec00 None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:53.137416 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df56ff-3abd-42dc-a3d4-2c03ca95ec00 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:53.137710 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df56ff-3abd-42dc-a3d4-2c03ca95ec00 None admin] PATH_INFO: `/v3/projects/b29240ce139e435497a96cfb6622dcc5/users/d820d6d390fa4ca188ab16ae1466c1df/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:53.143828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65df56ff-3abd-42dc-a3d4-2c03ca95ec00 None admin] RBAC: Authorizing `identity:create_grant(project_id=b29240ce139e435497a96cfb6622dcc5, user_id=d820d6d390fa4ca188ab16ae1466c1df, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:53.145304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65df56ff-3abd-42dc-a3d4-2c03ca95ec00 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:53.168127 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 70/142] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:52:52 2022] PUT /identity/v3/projects/b29240ce139e435497a96cfb6622dcc5/users/d820d6d390fa4ca188ab16ae1466c1df/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:52:53.169065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df56ff-3abd-42dc-a3d4-2c03ca95ec00 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:53.208506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:53.238509 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab485c7d-2133-4257-8c02-dc079c47428f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:53.247120 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:53.280503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8da59-7cd6-49d8-98c7-2ef886e26ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:53.281642 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8da59-7cd6-49d8-98c7-2ef886e26ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:53.282249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8da59-7cd6-49d8-98c7-2ef886e26ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:53.282785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca8da59-7cd6-49d8-98c7-2ef886e26ff5 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:53.283870 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca8da59-7cd6-49d8-98c7-2ef886e26ff5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:53.284484 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca8da59-7cd6-49d8-98c7-2ef886e26ff5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:53.286245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca8da59-7cd6-49d8-98c7-2ef886e26ff5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:53.296755 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 73/143] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:52:53 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:53.303179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8da59-7cd6-49d8-98c7-2ef886e26ff5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:53.308675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df56ff-3abd-42dc-a3d4-2c03ca95ec00 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:53.346499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64833a57-68ff-4729-8f52-d5816d225cba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:53.347316 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64833a57-68ff-4729-8f52-d5816d225cba None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:53.347613 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64833a57-68ff-4729-8f52-d5816d225cba None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:53.347899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64833a57-68ff-4729-8f52-d5816d225cba None admin] PATH_INFO: `/v3/domains/default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:53.349536 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64833a57-68ff-4729-8f52-d5816d225cba None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:53.350691 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64833a57-68ff-4729-8f52-d5816d225cba None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:53.352407 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 71/144] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:52:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:53.361816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64833a57-68ff-4729-8f52-d5816d225cba None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:53.459337 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca8da59-7cd6-49d8-98c7-2ef886e26ff5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:53.474167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64833a57-68ff-4729-8f52-d5816d225cba None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:53.505203 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c59cc-2640-4778-9f7a-3340e4a13a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:53.506501 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c59cc-2640-4778-9f7a-3340e4a13a35 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:53.506987 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c59cc-2640-4778-9f7a-3340e4a13a35 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:53.507511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8c59cc-2640-4778-9f7a-3340e4a13a35 None admin] PATH_INFO: `/v3/users/e549d830b8bf4a6d8e16e2dad340366b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:53.509040 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c59cc-2640-4778-9f7a-3340e4a13a35 None admin] RBAC: Authorizing `identity:get_user(user_id=e549d830b8bf4a6d8e16e2dad340366b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:53.510699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:53.512486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8c59cc-2640-4778-9f7a-3340e4a13a35 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:53.514551 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 74/145] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:52:53 2022] GET /identity/v3/users/e549d830b8bf4a6d8e16e2dad340366b => generated 247 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:53.521924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c59cc-2640-4778-9f7a-3340e4a13a35 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:53.540150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c7fc2-2f34-4c2a-a0ff-cebbc1d8188c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:53.540777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c7fc2-2f34-4c2a-a0ff-cebbc1d8188c None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:53.541069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c7fc2-2f34-4c2a-a0ff-cebbc1d8188c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:53.541353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f0c7fc2-2f34-4c2a-a0ff-cebbc1d8188c None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:53.542114 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c7fc2-2f34-4c2a-a0ff-cebbc1d8188c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:53.543100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:53.543878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f0c7fc2-2f34-4c2a-a0ff-cebbc1d8188c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:53.589262 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 72/146] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 11 13:52:53 2022] POST /identity/v3/users => generated 283 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:52:53.656497 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8c59cc-2640-4778-9f7a-3340e4a13a35 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:53.696813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9aba88-8e75-47c3-8119-1adfe0c36105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:53.697481 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aba88-8e75-47c3-8119-1adfe0c36105 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:53.697774 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aba88-8e75-47c3-8119-1adfe0c36105 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:53.698052 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9aba88-8e75-47c3-8119-1adfe0c36105 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:53.781550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9aba88-8e75-47c3-8119-1adfe0c36105 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:53.783117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9aba88-8e75-47c3-8119-1adfe0c36105 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:53.800871 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:53.965418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 75/147] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:52:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 444 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:52:53.972738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c7fc2-2f34-4c2a-a0ff-cebbc1d8188c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:54.165786 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f0c7fc2-2f34-4c2a-a0ff-cebbc1d8188c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:54.229908 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410efeaf-c547-4350-a33d-810e8e925a0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:54.231392 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410efeaf-c547-4350-a33d-810e8e925a0a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:54.231871 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410efeaf-c547-4350-a33d-810e8e925a0a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:54.232361 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410efeaf-c547-4350-a33d-810e8e925a0a None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:54.234404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410efeaf-c547-4350-a33d-810e8e925a0a None admin] RBAC: Authorizing `identity:get_project(project_id=9c32ce113f734d9cbd5e90555ab94cdd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:54.237728 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410efeaf-c547-4350-a33d-810e8e925a0a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:54.241064 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 73/148] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:52:53 2022] GET /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd => generated 298 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:52:54.253024 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9aba88-8e75-47c3-8119-1adfe0c36105 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:54.395216 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9aba88-8e75-47c3-8119-1adfe0c36105 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:54.421430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:54.436971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2a5b7e2-d3f6-4077-acb6-c414a1b289fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:54.437642 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5b7e2-d3f6-4077-acb6-c414a1b289fd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:54.437926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5b7e2-d3f6-4077-acb6-c414a1b289fd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:54.438241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a5b7e2-d3f6-4077-acb6-c414a1b289fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:54.439205 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5b7e2-d3f6-4077-acb6-c414a1b289fd None admin] RBAC: Adding query filter params (role.id=6964639a7f6c41e8a6701ef533151c41, scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd, user.id=e549d830b8bf4a6d8e16e2dad340366b) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:54.439519 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5b7e2-d3f6-4077-acb6-c414a1b289fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:54.440962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2a5b7e2-d3f6-4077-acb6-c414a1b289fd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:54.447459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 76/149] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:52:54 2022] GET /identity/v3/role_assignments?user.id=e549d830b8bf4a6d8e16e2dad340366b&scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 608 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:55.272568 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b4f2e-c73c-4069-99ec-8665a99e833b None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:55.273063 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b4f2e-c73c-4069-99ec-8665a99e833b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:55.273136 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d23b4f2e-c73c-4069-99ec-8665a99e833b None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:55.274907 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 74/150] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:55.282296 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caca621-42a1-4fb0-9e80-b098baa26b8b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:55.282296 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caca621-42a1-4fb0-9e80-b098baa26b8b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:55.282296 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4caca621-42a1-4fb0-9e80-b098baa26b8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:55.301208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4caca621-42a1-4fb0-9e80-b098baa26b8b None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:55.302805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4caca621-42a1-4fb0-9e80-b098baa26b8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:55.402473 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 77/151] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:55.439703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b4f2e-c73c-4069-99ec-8665a99e833b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:55.532478 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:55.589962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8fec2-cd07-4e85-9860-f5d4558ab9be None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:55.590296 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8fec2-cd07-4e85-9860-f5d4558ab9be None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:55.590581 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed8fec2-cd07-4e85-9860-f5d4558ab9be None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:55.592143 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 78/152] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:52:55.605198 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d23b4f2e-c73c-4069-99ec-8665a99e833b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:55.605492 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc773858-65e3-42cf-ab8a-fb92eb9e8613 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:55.605774 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc773858-65e3-42cf-ab8a-fb92eb9e8613 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:55.606061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc773858-65e3-42cf-ab8a-fb92eb9e8613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:55.637012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-dc773858-65e3-42cf-ab8a-fb92eb9e8613 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:55.638239 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dc773858-65e3-42cf-ab8a-fb92eb9e8613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:55.660184 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc66d02-acce-429d-989b-409b81057381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:55.661438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc66d02-acce-429d-989b-409b81057381 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:55.661725 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc66d02-acce-429d-989b-409b81057381 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:55.662316 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecc66d02-acce-429d-989b-409b81057381 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:55.685952 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecc66d02-acce-429d-989b-409b81057381 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:52:55.694449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-ecc66d02-acce-429d-989b-409b81057381 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:52:55.694449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:55.694449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:55.694449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:55.694449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:55.694449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:55.694449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:55.694449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:55.694449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:55.694449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:55.694449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:55.694449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:55.694449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:52:55.694449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:52:55.694449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:52:55.694449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:52:55.695421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:52:55.695421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:52:55.695421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:52:55.695421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:52:55.695421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:55.695421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:55.695421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:55.695421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:55.695421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:55.695421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:55.695421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:55.695421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:55.695421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:55.695421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:55.695421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:55.695421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:55.696932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:55.696932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:55.696932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:55.696932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:55.696932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:52:55.696932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:52:55.696932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:52:55.696932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:52:55.696932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:52:55.696932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:52:55.696932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:52:55.696932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:52:55.696932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 75/153] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:52:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:52:55.703292 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc66d02-acce-429d-989b-409b81057381 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:55.817332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 79/154] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:55.818750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5202717-f63d-44d6-a541-87745236a75f None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:55.820839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5202717-f63d-44d6-a541-87745236a75f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:55.820839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5202717-f63d-44d6-a541-87745236a75f None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:55.820839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 80/155] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:55.826105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd2ffd-6036-464b-9829-c53bc867cb79 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:55.826448 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd2ffd-6036-464b-9829-c53bc867cb79 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:55.826691 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17fd2ffd-6036-464b-9829-c53bc867cb79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:55.845820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-17fd2ffd-6036-464b-9829-c53bc867cb79 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:55.847122 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-17fd2ffd-6036-464b-9829-c53bc867cb79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:55.886356 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:55.938481 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecc66d02-acce-429d-989b-409b81057381 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:55.946213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 81/156] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:55.951316 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd2ffd-6036-464b-9829-c53bc867cb79 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:55.958334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:56.053116 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90781975-8423-470d-a863-cebc8d711f00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:56.054140 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90781975-8423-470d-a863-cebc8d711f00 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:56.054630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90781975-8423-470d-a863-cebc8d711f00 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:56.055311 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90781975-8423-470d-a863-cebc8d711f00 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:56.056278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90781975-8423-470d-a863-cebc8d711f00 None admin] RBAC: Adding query filter params (name=member) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:56.056747 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90781975-8423-470d-a863-cebc8d711f00 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:56.058399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90781975-8423-470d-a863-cebc8d711f00 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:56.100450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 76/157] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:52:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 398 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:56.101601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90781975-8423-470d-a863-cebc8d711f00 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:56.199012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17fd2ffd-6036-464b-9829-c53bc867cb79 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:56.307156 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea209b-eea3-4319-a7de-3ae2fe551125 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:56.308006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea209b-eea3-4319-a7de-3ae2fe551125 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:56.308327 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea209b-eea3-4319-a7de-3ae2fe551125 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:56.308557 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ea209b-eea3-4319-a7de-3ae2fe551125 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:56.334330 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90781975-8423-470d-a863-cebc8d711f00 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:56.339920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ea209b-eea3-4319-a7de-3ae2fe551125 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:52:56.349382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-65ea209b-eea3-4319-a7de-3ae2fe551125 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:52:56.349382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:56.349382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:56.349382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:56.349382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:56.349382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:56.349382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:56.349382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:56.349382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:56.349382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:56.349382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:56.349382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:56.349382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:52:56.349382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:52:56.349382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:52:56.349382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:52:56.350556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:52:56.350556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:52:56.350556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:52:56.350556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:52:56.350556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:56.350556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:56.350556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:56.350556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:56.350556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:56.350556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:56.350556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:56.350556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:56.350556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:56.350556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:56.350556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:56.350556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:56.351344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:56.351344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:56.351344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:56.351344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:56.351344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:52:56.351344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:52:56.351344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:52:56.351344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:52:56.351344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:52:56.351344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:52:56.351344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:52:56.351344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:52:56.351344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 82/158] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:52:55 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 404 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:52:56.356444 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea209b-eea3-4319-a7de-3ae2fe551125 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:56.382335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817db79b-6c4c-4921-a03f-378351706cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:56.382335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817db79b-6c4c-4921-a03f-378351706cec None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:56.382335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817db79b-6c4c-4921-a03f-378351706cec None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:56.382335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817db79b-6c4c-4921-a03f-378351706cec None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:56.414334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817db79b-6c4c-4921-a03f-378351706cec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:52:56.428632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-817db79b-6c4c-4921-a03f-378351706cec None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 13:52:56.428632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:56.428632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:56.428632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:56.428632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:56.428632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:56.428632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:56.428632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:56.428632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:56.428632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:56.428632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:56.428632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:56.428632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:52:56.428632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:52:56.428632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:52:56.428632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:52:56.429572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:52:56.429572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:52:56.429572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:52:56.429572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:52:56.429572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:56.429572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:56.429572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:56.429572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:56.429572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:56.429572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:56.429572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:56.429572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:56.429572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:56.429572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:56.429572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:56.429572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:56.430715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:56.430715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:56.430715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:56.430715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:56.430715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:52:56.430715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:52:56.430715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:52:56.430715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:52:56.430715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:52:56.430715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:52:56.430715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 13:52:56.430715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:52:56.433403 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 77/159] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:52:56 2022] GET /identity/v3/roles/reader => generated 84 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:52:56.435904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:56.437953 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4c13d-7034-4086-9f84-49addca50502 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:56.441108 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4c13d-7034-4086-9f84-49addca50502 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:56.441858 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cc4c13d-7034-4086-9f84-49addca50502 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:56.445108 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 78/160] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:56 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:56.446234 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc4c13d-7034-4086-9f84-49addca50502 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:56.530505 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:56.677225 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ea209b-eea3-4319-a7de-3ae2fe551125 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:56.743322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0873f-b5fb-4e38-884d-b49ae6f9d7c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:56.744217 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0873f-b5fb-4e38-884d-b49ae6f9d7c6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:56.744503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0873f-b5fb-4e38-884d-b49ae6f9d7c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:56.744779 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ee0873f-b5fb-4e38-884d-b49ae6f9d7c6 None admin] PATH_INFO: `/v3/users/d820d6d390fa4ca188ab16ae1466c1df` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:56.746459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cc4c13d-7034-4086-9f84-49addca50502 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:56.748419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0873f-b5fb-4e38-884d-b49ae6f9d7c6 None admin] RBAC: Authorizing `identity:get_user(user_id=d820d6d390fa4ca188ab16ae1466c1df)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:56.749889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ee0873f-b5fb-4e38-884d-b49ae6f9d7c6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:56.752226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 83/161] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:52:56 2022] GET /identity/v3/users/d820d6d390fa4ca188ab16ae1466c1df => generated 275 bytes in 401 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:56.759617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0873f-b5fb-4e38-884d-b49ae6f9d7c6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:56.815895 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ad824b8-73be-42a6-905f-47c8308b5d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:56.816492 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad824b8-73be-42a6-905f-47c8308b5d9c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:56.816778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad824b8-73be-42a6-905f-47c8308b5d9c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:56.817053 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ad824b8-73be-42a6-905f-47c8308b5d9c None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:56.817818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad824b8-73be-42a6-905f-47c8308b5d9c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:56.818127 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad824b8-73be-42a6-905f-47c8308b5d9c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:56.819536 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ad824b8-73be-42a6-905f-47c8308b5d9c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:56.849125 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 79/162] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:52:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:56.850698 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52536b4f-b732-4bc0-9e6d-2bbc98006a8c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:56.851253 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52536b4f-b732-4bc0-9e6d-2bbc98006a8c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:56.853912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52536b4f-b732-4bc0-9e6d-2bbc98006a8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:56.887782 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-52536b4f-b732-4bc0-9e6d-2bbc98006a8c None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:56.891564 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-52536b4f-b732-4bc0-9e6d-2bbc98006a8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:56.972168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:56.980842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ee0873f-b5fb-4e38-884d-b49ae6f9d7c6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:57.022549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32789c6-1edb-41ca-96c1-54c4a0dcb985 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:57.024314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32789c6-1edb-41ca-96c1-54c4a0dcb985 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:57.025949 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32789c6-1edb-41ca-96c1-54c4a0dcb985 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:57.027633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f32789c6-1edb-41ca-96c1-54c4a0dcb985 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:57.028558 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32789c6-1edb-41ca-96c1-54c4a0dcb985 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:57.029884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32789c6-1edb-41ca-96c1-54c4a0dcb985 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:57.031452 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f32789c6-1edb-41ca-96c1-54c4a0dcb985 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:57.046385 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 80/163] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:57.047547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52536b4f-b732-4bc0-9e6d-2bbc98006a8c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:57.051666 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 84/164] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:52:56 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 299 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:57.053046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32789c6-1edb-41ca-96c1-54c4a0dcb985 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:57.085547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:57.182590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f32789c6-1edb-41ca-96c1-54c4a0dcb985 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:57.229307 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd0a73-f590-4bce-962e-ce9e77342e9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:57.230202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd0a73-f590-4bce-962e-ce9e77342e9a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:57.230519 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd0a73-f590-4bce-962e-ce9e77342e9a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:57.230808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fd0a73-f590-4bce-962e-ce9e77342e9a None admin] PATH_INFO: `/v3/users/d820d6d390fa4ca188ab16ae1466c1df` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:57.234036 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd0a73-f590-4bce-962e-ce9e77342e9a None admin] RBAC: Authorizing `identity:get_user(user_id=d820d6d390fa4ca188ab16ae1466c1df)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:57.235576 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fd0a73-f590-4bce-962e-ce9e77342e9a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:57.237240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 85/165] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:52:57 2022] GET /identity/v3/users/d820d6d390fa4ca188ab16ae1466c1df => generated 275 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:57.238009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd0a73-f590-4bce-962e-ce9e77342e9a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:57.269256 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52536b4f-b732-4bc0-9e6d-2bbc98006a8c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:57.296610 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:57.345824 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e7f566-48ca-4934-bd26-005a3c332b85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:57.346425 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7f566-48ca-4934-bd26-005a3c332b85 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:57.346716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7f566-48ca-4934-bd26-005a3c332b85 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:57.346992 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e7f566-48ca-4934-bd26-005a3c332b85 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:57.434765 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fd0a73-f590-4bce-962e-ce9e77342e9a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:57.465844 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e7f566-48ca-4934-bd26-005a3c332b85 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:57.467418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36e7f566-48ca-4934-bd26-005a3c332b85 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:57.494210 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e8470-724d-4158-bf75-6a425872f1af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:57.495121 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e8470-724d-4158-bf75-6a425872f1af None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:57.495399 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e8470-724d-4158-bf75-6a425872f1af None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:57.495671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e8470-724d-4158-bf75-6a425872f1af None admin] PATH_INFO: `/v3/users/7d6a6792ad0141a1bcd2c420d03eb8fb` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:57.498028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e8470-724d-4158-bf75-6a425872f1af None admin] RBAC: Authorizing `identity:get_user(user_id=7d6a6792ad0141a1bcd2c420d03eb8fb)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:57.499506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e8470-724d-4158-bf75-6a425872f1af None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:57.501708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 86/166] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:52:57 2022] GET /identity/v3/users/7d6a6792ad0141a1bcd2c420d03eb8fb => generated 289 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:57.502495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e8470-724d-4158-bf75-6a425872f1af None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:57.504226 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:57.602973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 81/167] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:52:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 556 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:52:57.604158 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e7f566-48ca-4934-bd26-005a3c332b85 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:57.633283 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:57.642854 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e8470-724d-4158-bf75-6a425872f1af None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:57.681073 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b697a-3a4c-4094-bfae-4c8eb7fbad6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:57.682390 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b697a-3a4c-4094-bfae-4c8eb7fbad6c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:57.682878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b697a-3a4c-4094-bfae-4c8eb7fbad6c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:57.683353 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a73b697a-3a4c-4094-bfae-4c8eb7fbad6c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:57.695622 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a73b697a-3a4c-4094-bfae-4c8eb7fbad6c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:52:57.704639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a73b697a-3a4c-4094-bfae-4c8eb7fbad6c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:52:57.704639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:57.704639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:57.704639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:57.704639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:57.704639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:57.704639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:57.704639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:57.704639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:57.704639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:57.704639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:57.704639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:57.704639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:52:57.704639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:52:57.704639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:52:57.704639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:52:57.705568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:52:57.705568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:52:57.705568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:52:57.705568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:52:57.705568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:57.705568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:57.705568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:57.705568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:57.705568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:57.705568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:57.705568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:57.705568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:57.705568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:57.705568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:57.705568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:57.705568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:57.706376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:57.706376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:57.706376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:57.706376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:57.706376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:52:57.706376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:52:57.706376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:52:57.706376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:52:57.706376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:52:57.706376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:52:57.706376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:52:57.706376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:52:57.709065 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 87/168] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:52:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:52:57.710265 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b697a-3a4c-4094-bfae-4c8eb7fbad6c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:57.718492 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e7f566-48ca-4934-bd26-005a3c332b85 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:57.740249 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:57.758828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449579a-ab17-4602-ac19-afa1e314dfda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:57.759905 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449579a-ab17-4602-ac19-afa1e314dfda None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:57.760424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449579a-ab17-4602-ac19-afa1e314dfda None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:57.760734 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5449579a-ab17-4602-ac19-afa1e314dfda None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:57.835886 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a73b697a-3a4c-4094-bfae-4c8eb7fbad6c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:57.841352 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449579a-ab17-4602-ac19-afa1e314dfda None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:57.843154 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5449579a-ab17-4602-ac19-afa1e314dfda None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:57.878183 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9cad5b3-e6b9-4c92-8125-71229baf858d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:57.879573 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cad5b3-e6b9-4c92-8125-71229baf858d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:57.880061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cad5b3-e6b9-4c92-8125-71229baf858d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:57.880562 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9cad5b3-e6b9-4c92-8125-71229baf858d None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:57.882091 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cad5b3-e6b9-4c92-8125-71229baf858d None admin] RBAC: Authorizing `identity:get_project(project_id=9c32ce113f734d9cbd5e90555ab94cdd)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:57.884063 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9cad5b3-e6b9-4c92-8125-71229baf858d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:57.886620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 88/169] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:52:57 2022] GET /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:57.888256 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe4807-51ef-4284-9860-002816237297 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:57.888866 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe4807-51ef-4284-9860-002816237297 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:57.889343 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3abe4807-51ef-4284-9860-002816237297 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:57.891157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 89/170] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:57.892236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe4807-51ef-4284-9860-002816237297 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:57.917595 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:57.977487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 82/171] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:52:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:52:57.978788 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449579a-ab17-4602-ac19-afa1e314dfda None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:58.012148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:58.016437 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3abe4807-51ef-4284-9860-002816237297 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:58.058551 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472712d-06cb-4fde-afe9-be07066679a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:58.059720 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472712d-06cb-4fde-afe9-be07066679a4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:58.060283 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472712d-06cb-4fde-afe9-be07066679a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:58.060770 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c472712d-06cb-4fde-afe9-be07066679a4 None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:58.062258 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472712d-06cb-4fde-afe9-be07066679a4 None admin] RBAC: Authorizing `identity:get_project(project_id=9c32ce113f734d9cbd5e90555ab94cdd)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:58.064238 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c472712d-06cb-4fde-afe9-be07066679a4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:58.066244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 90/172] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:52:57 2022] GET /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd => generated 298 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:58.067465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472712d-06cb-4fde-afe9-be07066679a4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:58.119724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5449579a-ab17-4602-ac19-afa1e314dfda None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:58.157779 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fd36f1-f13e-46ed-8678-10946e3f99a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:58.158620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd36f1-f13e-46ed-8678-10946e3f99a1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:58.158911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd36f1-f13e-46ed-8678-10946e3f99a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:58.159190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fd36f1-f13e-46ed-8678-10946e3f99a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:58.159958 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd36f1-f13e-46ed-8678-10946e3f99a1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:58.160271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd36f1-f13e-46ed-8678-10946e3f99a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:58.161909 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fd36f1-f13e-46ed-8678-10946e3f99a1 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:58.174883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 83/173] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:52:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:58.174883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351c4fb-bdf8-4f33-b611-77e4f6f5bc70 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:58.174883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351c4fb-bdf8-4f33-b611-77e4f6f5bc70 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:58.174883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9351c4fb-bdf8-4f33-b611-77e4f6f5bc70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:58.190847 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:58.196053 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9351c4fb-bdf8-4f33-b611-77e4f6f5bc70 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:58.197459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9351c4fb-bdf8-4f33-b611-77e4f6f5bc70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:58.205024 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c472712d-06cb-4fde-afe9-be07066679a4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:58.240575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0a16f-9927-4151-9f0c-4dba2703bf4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:58.241463 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0a16f-9927-4151-9f0c-4dba2703bf4a None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:58.241745 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0a16f-9927-4151-9f0c-4dba2703bf4a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:58.242020 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a0a16f-9927-4151-9f0c-4dba2703bf4a None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd/users/7d6a6792ad0141a1bcd2c420d03eb8fb/roles/3a110b9d612545b09bf08554a2dd7eac` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:58.244122 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0a16f-9927-4151-9f0c-4dba2703bf4a None admin] RBAC: Authorizing `identity:create_grant(project_id=9c32ce113f734d9cbd5e90555ab94cdd, user_id=7d6a6792ad0141a1bcd2c420d03eb8fb, role_id=3a110b9d612545b09bf08554a2dd7eac)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:58.245561 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a0a16f-9927-4151-9f0c-4dba2703bf4a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:58.258243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 91/174] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:52:58 2022] PUT /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd/users/7d6a6792ad0141a1bcd2c420d03eb8fb/roles/3a110b9d612545b09bf08554a2dd7eac => generated 0 bytes in 191 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:52:58.259180 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0a16f-9927-4151-9f0c-4dba2703bf4a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:58.279445 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 84/175] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:58.289102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351c4fb-bdf8-4f33-b611-77e4f6f5bc70 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:58.307321 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:58.378118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a0a16f-9927-4151-9f0c-4dba2703bf4a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:58.422901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a7a1b3-6641-46c3-84f9-a70502af3df5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:58.422901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7a1b3-6641-46c3-84f9-a70502af3df5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:58.422901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7a1b3-6641-46c3-84f9-a70502af3df5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:58.422901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a7a1b3-6641-46c3-84f9-a70502af3df5 None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:58.423272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7a1b3-6641-46c3-84f9-a70502af3df5 None admin] RBAC: Authorizing `identity:get_project(project_id=9c32ce113f734d9cbd5e90555ab94cdd)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:58.423522 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a7a1b3-6641-46c3-84f9-a70502af3df5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:58.425503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 92/176] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:52:58 2022] GET /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd => generated 298 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:58.426806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1c374-0f7e-46d1-b93a-387e112c154c None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:58.427086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1c374-0f7e-46d1-b93a-387e112c154c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:58.427368 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f1c374-0f7e-46d1-b93a-387e112c154c None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:58.427464 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9351c4fb-bdf8-4f33-b611-77e4f6f5bc70 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:58.428820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 93/177] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:58.429366 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1c374-0f7e-46d1-b93a-387e112c154c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:58.449983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:58.467913 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ee3e38-338e-4de7-9521-719c76da6ed7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:58.468592 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee3e38-338e-4de7-9521-719c76da6ed7 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:58.468883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee3e38-338e-4de7-9521-719c76da6ed7 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:58.469178 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ee3e38-338e-4de7-9521-719c76da6ed7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:58.470172 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee3e38-338e-4de7-9521-719c76da6ed7 None admin] RBAC: Adding query filter params (role.id=827f1f1239754bf797e64f270ba1a80c, scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd, user.id=d820d6d390fa4ca188ab16ae1466c1df) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:58.470622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee3e38-338e-4de7-9521-719c76da6ed7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:58.472307 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ee3e38-338e-4de7-9521-719c76da6ed7 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:58.478745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 85/178] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:52:58 2022] GET /identity/v3/role_assignments?user.id=d820d6d390fa4ca188ab16ae1466c1df&scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd&role.id=827f1f1239754bf797e64f270ba1a80c => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:58.479817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065c350-ecca-4e18-8316-a97c24b018eb None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:58.480110 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065c350-ecca-4e18-8316-a97c24b018eb None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:58.480390 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3065c350-ecca-4e18-8316-a97c24b018eb None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:58.481820 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 86/179] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:52:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:52:58.486950 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065c350-ecca-4e18-8316-a97c24b018eb None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:58.578558 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f1c374-0f7e-46d1-b93a-387e112c154c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:58.610700 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:58.620471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1092e-c6bb-455c-ab62-c4dc0225d664 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:58.621627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1092e-c6bb-455c-ab62-c4dc0225d664 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:58.622090 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1092e-c6bb-455c-ab62-c4dc0225d664 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:58.622569 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c1092e-c6bb-455c-ab62-c4dc0225d664 None admin] PATH_INFO: `/v3/users/d820d6d390fa4ca188ab16ae1466c1df` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:58.624026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1092e-c6bb-455c-ab62-c4dc0225d664 None admin] RBAC: Authorizing `identity:get_user(user_id=d820d6d390fa4ca188ab16ae1466c1df)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:58.626348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c1092e-c6bb-455c-ab62-c4dc0225d664 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:58.628293 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 94/180] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:52:58 2022] GET /identity/v3/users/d820d6d390fa4ca188ab16ae1466c1df => generated 275 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:58.634074 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1092e-c6bb-455c-ab62-c4dc0225d664 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:58.647614 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3065c350-ecca-4e18-8316-a97c24b018eb None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:58.686525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682b99ba-a808-4c89-8992-c8f81fff0654 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:58.687264 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b99ba-a808-4c89-8992-c8f81fff0654 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:58.687557 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b99ba-a808-4c89-8992-c8f81fff0654 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:58.687861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682b99ba-a808-4c89-8992-c8f81fff0654 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:58.704431 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682b99ba-a808-4c89-8992-c8f81fff0654 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:52:58.712610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-682b99ba-a808-4c89-8992-c8f81fff0654 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:52:58.712610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:58.712610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:58.712610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:58.712610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:58.712610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:58.712610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:58.712610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:58.712610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:58.712610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:58.712610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:58.712610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:58.712610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:52:58.712610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:52:58.712610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:52:58.712610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:52:58.713703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:52:58.713703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:52:58.713703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:52:58.713703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:52:58.713703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:58.713703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:58.713703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:58.713703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:58.713703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:58.713703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:58.713703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:58.713703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:58.713703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:58.713703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:58.713703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:58.713703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:58.714737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:58.714737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:58.714737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:58.714737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:58.714737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:52:58.714737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:52:58.714737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:52:58.714737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:52:58.714737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:52:58.714737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:52:58.714737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:52:58.714737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:52:58.719853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 87/181] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:52:58 2022] GET /identity/v3/roles/admin => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:52:58.721273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64723e69-0ff5-41b1-9e6e-0f884df8f217 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:58.721521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64723e69-0ff5-41b1-9e6e-0f884df8f217 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:58.721803 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64723e69-0ff5-41b1-9e6e-0f884df8f217 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:58.740460 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-64723e69-0ff5-41b1-9e6e-0f884df8f217 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:58.741667 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-64723e69-0ff5-41b1-9e6e-0f884df8f217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:58.743478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:58.770528 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c1092e-c6bb-455c-ab62-c4dc0225d664 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:58.807563 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88595846-07d7-4b19-8ac7-5132700d09b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:58.808146 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88595846-07d7-4b19-8ac7-5132700d09b5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:58.808438 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88595846-07d7-4b19-8ac7-5132700d09b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:58.808719 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88595846-07d7-4b19-8ac7-5132700d09b5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:58.809599 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88595846-07d7-4b19-8ac7-5132700d09b5 None admin] RBAC: Adding query filter params (role.id=37f000302df54045892d35f57104660a, scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd, user.id=d820d6d390fa4ca188ab16ae1466c1df) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:58.809919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88595846-07d7-4b19-8ac7-5132700d09b5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:58.811435 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88595846-07d7-4b19-8ac7-5132700d09b5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:58.818699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 95/182] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:52:58 2022] GET /identity/v3/role_assignments?user.id=d820d6d390fa4ca188ab16ae1466c1df&scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd&role.id=37f000302df54045892d35f57104660a => generated 608 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:58.820239 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e5b0c-35b6-4d6b-ad75-15c69429aae5 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:58.820534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e5b0c-35b6-4d6b-ad75-15c69429aae5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:58.820813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95e5b0c-35b6-4d6b-ad75-15c69429aae5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:58.823750 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 88/183] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:58.824963 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64723e69-0ff5-41b1-9e6e-0f884df8f217 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:58.841330 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f95e5b0c-35b6-4d6b-ad75-15c69429aae5 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:52:58.843338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f95e5b0c-35b6-4d6b-ad75-15c69429aae5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:52:58.931100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:58.940198 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 96/184] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:52:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:52:58.941367 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e5b0c-35b6-4d6b-ad75-15c69429aae5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:58.963324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64723e69-0ff5-41b1-9e6e-0f884df8f217 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:59.026205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ef200-8d4e-4f9f-b6a4-0cac642ce6a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:59.026919 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ef200-8d4e-4f9f-b6a4-0cac642ce6a0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:59.027197 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ef200-8d4e-4f9f-b6a4-0cac642ce6a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:59.027477 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26ef200-8d4e-4f9f-b6a4-0cac642ce6a0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:59.035146 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:59.068170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95e5b0c-35b6-4d6b-ad75-15c69429aae5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:59.115009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29067a4c-8752-4eb1-8d92-64dc238e422c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:59.116515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29067a4c-8752-4eb1-8d92-64dc238e422c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:59.117278 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29067a4c-8752-4eb1-8d92-64dc238e422c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:59.117731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29067a4c-8752-4eb1-8d92-64dc238e422c None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:59.119363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29067a4c-8752-4eb1-8d92-64dc238e422c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:59.120029 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29067a4c-8752-4eb1-8d92-64dc238e422c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:59.121954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29067a4c-8752-4eb1-8d92-64dc238e422c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:59.133523 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 97/185] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:52:58 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:59.135024 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29067a4c-8752-4eb1-8d92-64dc238e422c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:59.135134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ef200-8d4e-4f9f-b6a4-0cac642ce6a0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:59.138295 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b26ef200-8d4e-4f9f-b6a4-0cac642ce6a0 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:59.213549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:59.246290 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29067a4c-8752-4eb1-8d92-64dc238e422c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:59.261514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 89/186] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:52:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 437 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:52:59.262432 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ef200-8d4e-4f9f-b6a4-0cac642ce6a0 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:59.288930 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d8699-63f2-4b56-b5ba-52ecdc618236 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:59.289716 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d8699-63f2-4b56-b5ba-52ecdc618236 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:59.290003 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d8699-63f2-4b56-b5ba-52ecdc618236 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:59.290279 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-107d8699-63f2-4b56-b5ba-52ecdc618236 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:59.291517 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d8699-63f2-4b56-b5ba-52ecdc618236 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:59.292991 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-107d8699-63f2-4b56-b5ba-52ecdc618236 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:59.294845 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 98/187] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:52:59 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:59.295405 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d8699-63f2-4b56-b5ba-52ecdc618236 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:59.309495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:59.382937 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26ef200-8d4e-4f9f-b6a4-0cac642ce6a0 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:59.405898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-107d8699-63f2-4b56-b5ba-52ecdc618236 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:59.424297 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a595f-6f01-450d-ae38-f2be87d0574d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:59.425311 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a595f-6f01-450d-ae38-f2be87d0574d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:59.425454 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a595f-6f01-450d-ae38-f2be87d0574d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:59.425682 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1a595f-6f01-450d-ae38-f2be87d0574d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:59.438173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d1a595f-6f01-450d-ae38-f2be87d0574d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:52:59.446429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-0d1a595f-6f01-450d-ae38-f2be87d0574d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:52:59.446429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:52:59.446429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:52:59.446429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:52:59.446429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:52:59.446429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:52:59.446429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:52:59.446429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:52:59.446429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:52:59.446429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:52:59.446429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:52:59.446429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:52:59.446429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:52:59.446429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:52:59.446429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:52:59.446429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:52:59.447313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:52:59.447313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:52:59.447313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:52:59.447313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:52:59.447313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:52:59.447313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:52:59.447313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:52:59.447313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:52:59.447313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:52:59.447313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:52:59.447313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:52:59.447313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:52:59.447313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:52:59.447313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:52:59.447313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:52:59.447313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:52:59.448103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:52:59.448103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:52:59.448103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:52:59.448103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:52:59.448103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:52:59.448103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:52:59.448103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:52:59.448103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:52:59.448103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:52:59.448103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:52:59.448103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:52:59.448103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:52:59.448103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 90/188] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:52:59 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:52:59.448829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a595f-6f01-450d-ae38-f2be87d0574d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:59.454529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8f8fc-611c-44ba-b9e5-d69b4751ad57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:59.454529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8f8fc-611c-44ba-b9e5-d69b4751ad57 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:59.454529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8f8fc-611c-44ba-b9e5-d69b4751ad57 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:59.454529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06e8f8fc-611c-44ba-b9e5-d69b4751ad57 None admin] PATH_INFO: `/v3/users/d0ccab2bdaae4f7ba089039a9f064a7b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:59.472918 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e8f8fc-611c-44ba-b9e5-d69b4751ad57 None admin] RBAC: Authorizing `identity:get_user(user_id=d0ccab2bdaae4f7ba089039a9f064a7b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:59.474346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06e8f8fc-611c-44ba-b9e5-d69b4751ad57 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:59.476100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 99/189] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:52:59 2022] GET /identity/v3/users/d0ccab2bdaae4f7ba089039a9f064a7b => generated 283 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:59.476934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8f8fc-611c-44ba-b9e5-d69b4751ad57 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:59.486342 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:59.554026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:59.577647 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1a595f-6f01-450d-ae38-f2be87d0574d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:59.592696 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06e8f8fc-611c-44ba-b9e5-d69b4751ad57 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:59.617310 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6a5c2-ed53-4291-b6ef-4d1f87baded4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:59.618185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6a5c2-ed53-4291-b6ef-4d1f87baded4 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:59.618500 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6a5c2-ed53-4291-b6ef-4d1f87baded4 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:59.618776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f6a5c2-ed53-4291-b6ef-4d1f87baded4 None admin] PATH_INFO: `/v3/projects/b29240ce139e435497a96cfb6622dcc5` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:59.619988 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6a5c2-ed53-4291-b6ef-4d1f87baded4 None admin] RBAC: Authorizing `identity:get_project(project_id=b29240ce139e435497a96cfb6622dcc5)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:59.621441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f6a5c2-ed53-4291-b6ef-4d1f87baded4 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:59.623356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 91/190] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:52:59 2022] GET /identity/v3/projects/b29240ce139e435497a96cfb6622dcc5 => generated 312 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:59.624102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6a5c2-ed53-4291-b6ef-4d1f87baded4 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:59.636590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362cc2c-bcbb-41e9-af0d-a7b37a504138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:59.637393 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362cc2c-bcbb-41e9-af0d-a7b37a504138 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:59.637684 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362cc2c-bcbb-41e9-af0d-a7b37a504138 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:59.637964 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6362cc2c-bcbb-41e9-af0d-a7b37a504138 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:59.638746 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362cc2c-bcbb-41e9-af0d-a7b37a504138 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:59.640188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6362cc2c-bcbb-41e9-af0d-a7b37a504138 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:59.663384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 100/191] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 11 13:52:59 2022] POST /identity/v3/users => generated 297 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:52:59.664231 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362cc2c-bcbb-41e9-af0d-a7b37a504138 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:59.752250 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:59.765483 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f6a5c2-ed53-4291-b6ef-4d1f87baded4 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:59.798389 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6362cc2c-bcbb-41e9-af0d-a7b37a504138 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:52:59.819727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bb239-0f90-45ce-b1b4-c716356868fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:59.820891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bb239-0f90-45ce-b1b4-c716356868fe None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:59.821304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bb239-0f90-45ce-b1b4-c716356868fe None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:59.821721 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-407bb239-0f90-45ce-b1b4-c716356868fe None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:59.823103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407bb239-0f90-45ce-b1b4-c716356868fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:52:59.823533 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407bb239-0f90-45ce-b1b4-c716356868fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:59.825124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-407bb239-0f90-45ce-b1b4-c716356868fe None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:59.831803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:52:59.843069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 92/192] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:52:59 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:52:59.845149 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bb239-0f90-45ce-b1b4-c716356868fe None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:52:59.855670 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378022c6-cc0d-409d-bc51-7299c7407edd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:52:59.856590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378022c6-cc0d-409d-bc51-7299c7407edd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:52:59.857034 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378022c6-cc0d-409d-bc51-7299c7407edd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:52:59.857317 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378022c6-cc0d-409d-bc51-7299c7407edd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:52:59.955286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378022c6-cc0d-409d-bc51-7299c7407edd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:52:59.958847 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378022c6-cc0d-409d-bc51-7299c7407edd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:52:59.976577 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-407bb239-0f90-45ce-b1b4-c716356868fe None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:00.024318 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce2a4a-1785-4a66-a587-acd19ad7e2fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:00.025359 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce2a4a-1785-4a66-a587-acd19ad7e2fd None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:00.025652 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce2a4a-1785-4a66-a587-acd19ad7e2fd None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:00.025937 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ce2a4a-1785-4a66-a587-acd19ad7e2fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:00.027140 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce2a4a-1785-4a66-a587-acd19ad7e2fd None admin] RBAC: Adding query filter params (role.id=827f1f1239754bf797e64f270ba1a80c, scope.project.id=b29240ce139e435497a96cfb6622dcc5, user.id=d820d6d390fa4ca188ab16ae1466c1df) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:00.027436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce2a4a-1785-4a66-a587-acd19ad7e2fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:00.029397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ce2a4a-1785-4a66-a587-acd19ad7e2fd None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:00.036571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 93/193] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:52:59 2022] GET /identity/v3/role_assignments?user.id=d820d6d390fa4ca188ab16ae1466c1df&scope.project.id=b29240ce139e435497a96cfb6622dcc5&role.id=827f1f1239754bf797e64f270ba1a80c => generated 608 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:00.043276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce2a4a-1785-4a66-a587-acd19ad7e2fd None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:00.086045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:00.102900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 101/194] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:52:59 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 439 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:00.109711 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378022c6-cc0d-409d-bc51-7299c7407edd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:00.145522 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:00.196463 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ce2a4a-1785-4a66-a587-acd19ad7e2fd None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:00.238230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91475e-6670-48b6-9a11-c8f1ec223793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:00.239194 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91475e-6670-48b6-9a11-c8f1ec223793 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:00.239453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378022c6-cc0d-409d-bc51-7299c7407edd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:00.239584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91475e-6670-48b6-9a11-c8f1ec223793 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:00.239754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec91475e-6670-48b6-9a11-c8f1ec223793 None admin] PATH_INFO: `/v3/users/d0ccab2bdaae4f7ba089039a9f064a7b` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:00.241114 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec91475e-6670-48b6-9a11-c8f1ec223793 None admin] RBAC: Authorizing `identity:get_user(user_id=d0ccab2bdaae4f7ba089039a9f064a7b)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:00.242715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec91475e-6670-48b6-9a11-c8f1ec223793 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:00.244654 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 94/195] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:53:00 2022] GET /identity/v3/users/d0ccab2bdaae4f7ba089039a9f064a7b => generated 283 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:00.251104 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91475e-6670-48b6-9a11-c8f1ec223793 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:00.284553 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3971d-3490-4625-8550-928df5b3b12e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:00.285776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3971d-3490-4625-8550-928df5b3b12e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:00.286284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3971d-3490-4625-8550-928df5b3b12e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:00.287004 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd3971d-3490-4625-8550-928df5b3b12e None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:00.289097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3971d-3490-4625-8550-928df5b3b12e None admin] RBAC: Authorizing `identity:get_project(project_id=34304bd3ae654c1c976f55ca788ae592)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:00.291280 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fd3971d-3490-4625-8550-928df5b3b12e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:00.296421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 102/196] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:00 2022] GET /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:00.305304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3971d-3490-4625-8550-928df5b3b12e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:00.417339 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:00.457541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec91475e-6670-48b6-9a11-c8f1ec223793 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:00.497747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:00.527412 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd3971d-3490-4625-8550-928df5b3b12e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:00.548995 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416abc76-dda2-4d61-9654-c98b49d27d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:00.550015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416abc76-dda2-4d61-9654-c98b49d27d7c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:00.551398 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416abc76-dda2-4d61-9654-c98b49d27d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:00.551398 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416abc76-dda2-4d61-9654-c98b49d27d7c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:00.598074 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e3fb30-fa7a-4a24-8ddc-26f7380e506c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:00.599203 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3fb30-fa7a-4a24-8ddc-26f7380e506c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:00.599713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3fb30-fa7a-4a24-8ddc-26f7380e506c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:00.600210 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e3fb30-fa7a-4a24-8ddc-26f7380e506c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:00.601911 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3fb30-fa7a-4a24-8ddc-26f7380e506c None admin] RBAC: Adding query filter params (role.id=6964639a7f6c41e8a6701ef533151c41, scope.project.id=34304bd3ae654c1c976f55ca788ae592, user.id=d0ccab2bdaae4f7ba089039a9f064a7b) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:00.602526 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3fb30-fa7a-4a24-8ddc-26f7380e506c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:00.605261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e3fb30-fa7a-4a24-8ddc-26f7380e506c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:00.617411 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 103/197] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:53:00 2022] GET /identity/v3/role_assignments?user.id=d0ccab2bdaae4f7ba089039a9f064a7b&scope.project.id=34304bd3ae654c1c976f55ca788ae592&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 259 bytes in 313 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:00.619935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c2d3d-1339-4ee0-bc0a-6887f40c8a26 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:00.620225 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c2d3d-1339-4ee0-bc0a-6887f40c8a26 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:00.620659 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0c2d3d-1339-4ee0-bc0a-6887f40c8a26 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:00.626426 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 104/198] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:00.636473 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bab0d-1392-4e7e-822d-30c37fc4ece3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:00.637286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bab0d-1392-4e7e-822d-30c37fc4ece3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:00.637869 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0bab0d-1392-4e7e-822d-30c37fc4ece3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:00.666639 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416abc76-dda2-4d61-9654-c98b49d27d7c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:00.668166 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-416abc76-dda2-4d61-9654-c98b49d27d7c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:00.695529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8e0bab0d-1392-4e7e-822d-30c37fc4ece3 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:00.696784 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8e0bab0d-1392-4e7e-822d-30c37fc4ece3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:00.784420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:00.810069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 95/199] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 558 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 13:53:00.819571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416abc76-dda2-4d61-9654-c98b49d27d7c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:00.863757 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 105/200] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:00.889244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bab0d-1392-4e7e-822d-30c37fc4ece3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:01.023312 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416abc76-dda2-4d61-9654-c98b49d27d7c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:01.038681 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:01.072635 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0bab0d-1392-4e7e-822d-30c37fc4ece3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:01.105845 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a305d-d384-45d3-8555-7158b2233c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:01.106774 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a305d-d384-45d3-8555-7158b2233c8c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:01.107060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a305d-d384-45d3-8555-7158b2233c8c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:01.107330 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3a305d-d384-45d3-8555-7158b2233c8c None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:01.108774 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a305d-d384-45d3-8555-7158b2233c8c None admin] RBAC: Authorizing `identity:get_project(project_id=34304bd3ae654c1c976f55ca788ae592)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:01.110321 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3a305d-d384-45d3-8555-7158b2233c8c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:01.112112 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 96/201] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:00 2022] GET /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592 => generated 302 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:01.119972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a305d-d384-45d3-8555-7158b2233c8c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:01.151826 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615c5ad-4792-482e-9f8e-3503c9b5b055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:01.152654 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615c5ad-4792-482e-9f8e-3503c9b5b055 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:01.152937 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615c5ad-4792-482e-9f8e-3503c9b5b055 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:01.153214 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6615c5ad-4792-482e-9f8e-3503c9b5b055 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:01.181099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6615c5ad-4792-482e-9f8e-3503c9b5b055 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:01.202118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-6615c5ad-4792-482e-9f8e-3503c9b5b055 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 13:53:01.202118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:01.202118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:01.202118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:01.202118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:01.202118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:01.202118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:01.202118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:01.202118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:01.202118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:01.202118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:01.202118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:01.202118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:01.202118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:01.202118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:01.202118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:01.203124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:01.203124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:01.203124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:01.203124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:01.203124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:01.203124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:01.203124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:01.203124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:01.203124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:01.203124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:01.203124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:01.203124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:01.203124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:01.203124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:01.203124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:01.203124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:01.204454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:01.204454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:01.204454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:01.204454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:01.204454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:01.204454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:01.204454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:01.204454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:01.204454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:01.204454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:01.204454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 13:53:01.204454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:01.210917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 106/202] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 322 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:53:01.227820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615c5ad-4792-482e-9f8e-3503c9b5b055 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:01.244287 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:01.312997 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3a305d-d384-45d3-8555-7158b2233c8c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:01.357683 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef58ce-e486-4aad-b408-0e90f8ac58d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:01.358277 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef58ce-e486-4aad-b408-0e90f8ac58d5 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:01.358584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef58ce-e486-4aad-b408-0e90f8ac58d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:01.358863 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71ef58ce-e486-4aad-b408-0e90f8ac58d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:01.359723 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef58ce-e486-4aad-b408-0e90f8ac58d5 None admin] RBAC: Adding query filter params (role.id=37f000302df54045892d35f57104660a, scope.project.id=34304bd3ae654c1c976f55ca788ae592, user.id=d0ccab2bdaae4f7ba089039a9f064a7b) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:01.360089 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef58ce-e486-4aad-b408-0e90f8ac58d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:01.361571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71ef58ce-e486-4aad-b408-0e90f8ac58d5 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:01.377083 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 97/203] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:53:01 2022] GET /identity/v3/role_assignments?user.id=d0ccab2bdaae4f7ba089039a9f064a7b&scope.project.id=34304bd3ae654c1c976f55ca788ae592&role.id=37f000302df54045892d35f57104660a => generated 259 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:01.426090 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6615c5ad-4792-482e-9f8e-3503c9b5b055 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:01.491741 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad13efb-2640-46c5-8fdf-a2e5a8d007fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:01.492442 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad13efb-2640-46c5-8fdf-a2e5a8d007fa None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:01.492728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad13efb-2640-46c5-8fdf-a2e5a8d007fa None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:01.493008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad13efb-2640-46c5-8fdf-a2e5a8d007fa None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:01.493818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad13efb-2640-46c5-8fdf-a2e5a8d007fa None admin] RBAC: Adding query filter params (name=reader) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:01.494129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad13efb-2640-46c5-8fdf-a2e5a8d007fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:01.495638 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad13efb-2640-46c5-8fdf-a2e5a8d007fa None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:01.506740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 107/204] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:53:01 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:01.512502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef58ce-e486-4aad-b408-0e90f8ac58d5 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:01.633310 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71ef58ce-e486-4aad-b408-0e90f8ac58d5 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:01.667825 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:01.690382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dbf0e-14f3-4b6a-878a-5f90ce8af903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:01.691277 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dbf0e-14f3-4b6a-878a-5f90ce8af903 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:01.691563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dbf0e-14f3-4b6a-878a-5f90ce8af903 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:01.691911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe8dbf0e-14f3-4b6a-878a-5f90ce8af903 None admin] PATH_INFO: `/v3/users/7d6a6792ad0141a1bcd2c420d03eb8fb` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:01.697899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8dbf0e-14f3-4b6a-878a-5f90ce8af903 None admin] RBAC: Authorizing `identity:get_user(user_id=7d6a6792ad0141a1bcd2c420d03eb8fb)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:01.699545 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe8dbf0e-14f3-4b6a-878a-5f90ce8af903 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:01.701422 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 98/205] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:53:01 2022] GET /identity/v3/users/7d6a6792ad0141a1bcd2c420d03eb8fb => generated 289 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:01.706984 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad13efb-2640-46c5-8fdf-a2e5a8d007fa None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:01.716901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:01.824969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad13efb-2640-46c5-8fdf-a2e5a8d007fa None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:01.862135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474526f-cfe6-4ec1-a9bb-5626a3e373c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:01.862905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474526f-cfe6-4ec1-a9bb-5626a3e373c6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:01.863197 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474526f-cfe6-4ec1-a9bb-5626a3e373c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:01.863480 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1474526f-cfe6-4ec1-a9bb-5626a3e373c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:01.943492 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474526f-cfe6-4ec1-a9bb-5626a3e373c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:01.945744 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1474526f-cfe6-4ec1-a9bb-5626a3e373c6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:01.955449 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:02.083271 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 108/206] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:02.089164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dbf0e-14f3-4b6a-878a-5f90ce8af903 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:02.227371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe8dbf0e-14f3-4b6a-878a-5f90ce8af903 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:02.278002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5989bc7-a9bc-4dcd-ab3e-79d5f8cee5f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:02.278928 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5989bc7-a9bc-4dcd-ab3e-79d5f8cee5f3 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:02.279216 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5989bc7-a9bc-4dcd-ab3e-79d5f8cee5f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:02.279495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5989bc7-a9bc-4dcd-ab3e-79d5f8cee5f3 None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:02.280749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5989bc7-a9bc-4dcd-ab3e-79d5f8cee5f3 None admin] RBAC: Authorizing `identity:get_project(project_id=9c32ce113f734d9cbd5e90555ab94cdd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:02.282816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5989bc7-a9bc-4dcd-ab3e-79d5f8cee5f3 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:02.284565 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 99/207] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:02 2022] GET /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:02.290695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474526f-cfe6-4ec1-a9bb-5626a3e373c6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:02.436487 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1474526f-cfe6-4ec1-a9bb-5626a3e373c6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:02.449017 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:02.486781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35de752-509d-4742-8809-d94a1e002224 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:02.487453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35de752-509d-4742-8809-d94a1e002224 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:02.487738 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35de752-509d-4742-8809-d94a1e002224 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:02.488018 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35de752-509d-4742-8809-d94a1e002224 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:02.488871 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35de752-509d-4742-8809-d94a1e002224 None admin] RBAC: Adding query filter params (role.id=3a110b9d612545b09bf08554a2dd7eac, scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd, user.id=7d6a6792ad0141a1bcd2c420d03eb8fb) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:02.489187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35de752-509d-4742-8809-d94a1e002224 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:02.490650 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35de752-509d-4742-8809-d94a1e002224 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:02.502446 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 109/208] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:53:02 2022] GET /identity/v3/role_assignments?user.id=7d6a6792ad0141a1bcd2c420d03eb8fb&scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd&role.id=3a110b9d612545b09bf08554a2dd7eac => generated 608 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:02.759596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64007f7a-a467-4395-9d1d-827f037f49a1 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:02.760078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64007f7a-a467-4395-9d1d-827f037f49a1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:02.760078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64007f7a-a467-4395-9d1d-827f037f49a1 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:02.761523 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 100/209] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:02.769066 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ad46c-5812-4fb4-93f3-035d4b466264 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:02.770053 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ad46c-5812-4fb4-93f3-035d4b466264 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:02.770577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ad46c-5812-4fb4-93f3-035d4b466264 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:02.793427 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7c2ad46c-5812-4fb4-93f3-035d4b466264 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:02.795057 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7c2ad46c-5812-4fb4-93f3-035d4b466264 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:02.891723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 110/210] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:02.916557 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64007f7a-a467-4395-9d1d-827f037f49a1 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:02.943947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b997192-f2c4-4371-993b-2eba555444f1 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:02.944393 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b997192-f2c4-4371-993b-2eba555444f1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:02.944794 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b997192-f2c4-4371-993b-2eba555444f1 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:02.947410 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 111/211] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:02 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:02.957589 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70f99b-1381-4db8-a046-1def857ed149 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:02.958209 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70f99b-1381-4db8-a046-1def857ed149 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:02.958973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc70f99b-1381-4db8-a046-1def857ed149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:02.983082 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cc70f99b-1381-4db8-a046-1def857ed149 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:02.984876 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cc70f99b-1381-4db8-a046-1def857ed149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:03.040770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:03.095350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 112/212] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:53:03.118266 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64007f7a-a467-4395-9d1d-827f037f49a1 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:03.144844 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70f99b-1381-4db8-a046-1def857ed149 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:03.174093 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c3b35-668b-448c-8027-db2345016136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:03.175360 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c3b35-668b-448c-8027-db2345016136 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:03.175885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c3b35-668b-448c-8027-db2345016136 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:03.176434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255c3b35-668b-448c-8027-db2345016136 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:03.193608 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255c3b35-668b-448c-8027-db2345016136 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:03.200255 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:03.207853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-255c3b35-668b-448c-8027-db2345016136 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:03.207853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:03.207853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:03.207853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:03.207853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:03.207853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:03.207853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:03.207853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:03.207853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:03.207853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:03.207853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:03.207853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:03.207853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:03.207853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:03.207853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:03.207853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:03.209457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:03.209457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:03.209457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:03.209457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:03.209457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:03.209457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:03.209457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:03.209457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:03.209457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:03.209457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:03.209457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:03.209457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:03.209457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:03.209457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:03.209457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:03.209457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:03.210986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:03.210986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:03.210986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:03.210986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:03.210986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:03.210986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:03.210986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:03.210986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:03.210986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:03.210986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:03.210986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:03.210986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:03.214754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 101/213] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:03.226357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c3b35-668b-448c-8027-db2345016136 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:03.316709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc70f99b-1381-4db8-a046-1def857ed149 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:03.361317 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bcf228-f918-4385-a580-ccbb52616ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:03.362352 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcf228-f918-4385-a580-ccbb52616ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:03.362352 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcf228-f918-4385-a580-ccbb52616ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:03.362352 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bcf228-f918-4385-a580-ccbb52616ed3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:03.363568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcf228-f918-4385-a580-ccbb52616ed3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:03.365037 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9bcf228-f918-4385-a580-ccbb52616ed3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:03.366773 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 113/214] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:03 2022] GET /identity/v3/domains/default => generated 199 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:03.368075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dab1bc-1bf5-40a1-8c6c-1b72cae4338b None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:03.373341 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dab1bc-1bf5-40a1-8c6c-1b72cae4338b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:03.373686 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dab1bc-1bf5-40a1-8c6c-1b72cae4338b None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:03.375994 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 114/215] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:03 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:03.380875 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dab1bc-1bf5-40a1-8c6c-1b72cae4338b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:03.389526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255c3b35-668b-448c-8027-db2345016136 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:03.412623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:03.445363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e508278b-6468-4d76-967e-720cf908aa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:03.446151 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508278b-6468-4d76-967e-720cf908aa07 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:03.446490 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508278b-6468-4d76-967e-720cf908aa07 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:03.446749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e508278b-6468-4d76-967e-720cf908aa07 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:03.447532 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508278b-6468-4d76-967e-720cf908aa07 None admin] RBAC: Adding query filter params (name=member) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:03.447845 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508278b-6468-4d76-967e-720cf908aa07 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:03.449286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e508278b-6468-4d76-967e-720cf908aa07 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:03.459955 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 102/216] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:53:03 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:03.461123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764faf45-2475-4a68-9ac4-f7181c5a385d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:03.461427 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764faf45-2475-4a68-9ac4-f7181c5a385d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:03.461701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-764faf45-2475-4a68-9ac4-f7181c5a385d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:03.481550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-764faf45-2475-4a68-9ac4-f7181c5a385d None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:03.482789 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-764faf45-2475-4a68-9ac4-f7181c5a385d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:03.529615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:03.538086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dab1bc-1bf5-40a1-8c6c-1b72cae4338b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:03.564038 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 103/217] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:03.565145 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764faf45-2475-4a68-9ac4-f7181c5a385d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:03.592200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c8883-33af-475f-a946-1fa9bc47573b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:03.593234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c8883-33af-475f-a946-1fa9bc47573b None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:03.593712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c8883-33af-475f-a946-1fa9bc47573b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:03.593994 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54c8883-33af-475f-a946-1fa9bc47573b None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:03.595380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c8883-33af-475f-a946-1fa9bc47573b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:03.596880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54c8883-33af-475f-a946-1fa9bc47573b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:03.619949 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 115/218] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 11 13:53:03 2022] POST /identity/v3/users => generated 297 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:53:03.620908 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c8883-33af-475f-a946-1fa9bc47573b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:03.716877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-764faf45-2475-4a68-9ac4-f7181c5a385d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:03.730016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:03.749994 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54c8883-33af-475f-a946-1fa9bc47573b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:03.779197 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f4f1e-84c0-463e-ab7e-87df60abc2f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:03.781454 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f4f1e-84c0-463e-ab7e-87df60abc2f4 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:03.782085 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f4f1e-84c0-463e-ab7e-87df60abc2f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:03.782565 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87f4f1e-84c0-463e-ab7e-87df60abc2f4 None admin] PATH_INFO: `/v3/users/05bf542db18b415087bef3ca1a0395c8` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:03.795148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9647ed9-329c-45be-9e96-f36fe9bbe0f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:03.796282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9647ed9-329c-45be-9e96-f36fe9bbe0f8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:03.796666 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9647ed9-329c-45be-9e96-f36fe9bbe0f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:03.796856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9647ed9-329c-45be-9e96-f36fe9bbe0f8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:03.808540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f4f1e-84c0-463e-ab7e-87df60abc2f4 None admin] RBAC: Authorizing `identity:get_user(user_id=05bf542db18b415087bef3ca1a0395c8)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:03.812560 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87f4f1e-84c0-463e-ab7e-87df60abc2f4 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:03.816197 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9647ed9-329c-45be-9e96-f36fe9bbe0f8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:03.817768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 104/219] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:53:03 2022] GET /identity/v3/users/05bf542db18b415087bef3ca1a0395c8 => generated 297 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:03.827228 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f4f1e-84c0-463e-ab7e-87df60abc2f4 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:03.829400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-d9647ed9-329c-45be-9e96-f36fe9bbe0f8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:53:03.829400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:03.829400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:03.829400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:03.829400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:03.829400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:03.829400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:03.829400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:03.829400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:03.829400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:03.829400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:03.829400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:03.829400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:03.829400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:03.829400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:03.829400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:03.830808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:03.830808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:03.830808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:03.830808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:03.830808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:03.830808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:03.830808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:03.830808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:03.830808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:03.830808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:03.830808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:03.830808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:03.830808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:03.830808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:03.830808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:03.830808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:03.832080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:03.832080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:03.832080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:03.832080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:03.832080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:03.832080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:03.832080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:03.832080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:03.832080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:03.832080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:03.832080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:53:03.832080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:03.832080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 116/220] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:53:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:03.836793 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9647ed9-329c-45be-9e96-f36fe9bbe0f8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:03.858720 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:03.985532 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9647ed9-329c-45be-9e96-f36fe9bbe0f8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:03.995400 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87f4f1e-84c0-463e-ab7e-87df60abc2f4 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:04.035362 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e63285-4acd-48b6-88e9-f4f6ecf0a6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:04.036058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e63285-4acd-48b6-88e9-f4f6ecf0a6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:04.036379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e63285-4acd-48b6-88e9-f4f6ecf0a6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:04.036640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e63285-4acd-48b6-88e9-f4f6ecf0a6b0 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:04.037406 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e63285-4acd-48b6-88e9-f4f6ecf0a6b0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:04.037723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e63285-4acd-48b6-88e9-f4f6ecf0a6b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:04.040301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e63285-4acd-48b6-88e9-f4f6ecf0a6b0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:04.041890 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bbac93a-60c4-47e1-8475-9e87d6807433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:04.043003 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbac93a-60c4-47e1-8475-9e87d6807433 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:04.043294 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbac93a-60c4-47e1-8475-9e87d6807433 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:04.043575 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bbac93a-60c4-47e1-8475-9e87d6807433 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:04.049980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 117/221] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:04.054731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995c81d-6705-4e73-bae0-e554c4a0474d None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:04.054731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995c81d-6705-4e73-bae0-e554c4a0474d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:04.055195 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4995c81d-6705-4e73-bae0-e554c4a0474d None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:04.057937 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 118/222] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:04.060887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995c81d-6705-4e73-bae0-e554c4a0474d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:04.088109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:04.135797 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbac93a-60c4-47e1-8475-9e87d6807433 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:04.138359 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bbac93a-60c4-47e1-8475-9e87d6807433 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:04.171859 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:04.203096 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4995c81d-6705-4e73-bae0-e554c4a0474d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:04.247361 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 105/223] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 13:53:04.248916 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7017688-7a86-4cc1-a40c-83fc3af98c55 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:04.249461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7017688-7a86-4cc1-a40c-83fc3af98c55 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:04.249934 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7017688-7a86-4cc1-a40c-83fc3af98c55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:04.253680 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18e2cc-f808-45e9-8360-c5c9408d464c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:04.254597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18e2cc-f808-45e9-8360-c5c9408d464c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:04.254881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18e2cc-f808-45e9-8360-c5c9408d464c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:04.255162 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b18e2cc-f808-45e9-8360-c5c9408d464c None admin] PATH_INFO: `/v3/users/d0ccab2bdaae4f7ba089039a9f064a7b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:04.256406 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18e2cc-f808-45e9-8360-c5c9408d464c None admin] RBAC: Authorizing `identity:get_user(user_id=d0ccab2bdaae4f7ba089039a9f064a7b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:04.258126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b18e2cc-f808-45e9-8360-c5c9408d464c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:04.262378 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 119/224] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:53:04 2022] GET /identity/v3/users/d0ccab2bdaae4f7ba089039a9f064a7b => generated 283 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:04.262378 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18e2cc-f808-45e9-8360-c5c9408d464c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:04.280525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c7017688-7a86-4cc1-a40c-83fc3af98c55 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:04.281847 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c7017688-7a86-4cc1-a40c-83fc3af98c55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:04.352124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:04.352442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 106/225] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:04.354034 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7017688-7a86-4cc1-a40c-83fc3af98c55 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:04.381262 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b18e2cc-f808-45e9-8360-c5c9408d464c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:04.428642 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58216d2a-5825-4ee9-ae7c-03887c091341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:04.430763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58216d2a-5825-4ee9-ae7c-03887c091341 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:04.431130 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58216d2a-5825-4ee9-ae7c-03887c091341 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:04.431486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58216d2a-5825-4ee9-ae7c-03887c091341 None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:04.432848 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58216d2a-5825-4ee9-ae7c-03887c091341 None admin] RBAC: Authorizing `identity:get_project(project_id=34304bd3ae654c1c976f55ca788ae592)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:04.434420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58216d2a-5825-4ee9-ae7c-03887c091341 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:04.436410 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 120/226] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:04 2022] GET /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592 => generated 302 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:04.437103 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58216d2a-5825-4ee9-ae7c-03887c091341 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:04.462145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:04.471690 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7017688-7a86-4cc1-a40c-83fc3af98c55 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:04.517245 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1667c3-f4e3-454f-a1dc-d338e0226046 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:04.518519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1667c3-f4e3-454f-a1dc-d338e0226046 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:04.518897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1667c3-f4e3-454f-a1dc-d338e0226046 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:04.519291 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca1667c3-f4e3-454f-a1dc-d338e0226046 None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:04.521315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1667c3-f4e3-454f-a1dc-d338e0226046 None admin] RBAC: Authorizing `identity:get_project(project_id=34304bd3ae654c1c976f55ca788ae592)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:04.523112 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca1667c3-f4e3-454f-a1dc-d338e0226046 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:04.524866 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 107/227] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:04 2022] GET /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:04.525642 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1667c3-f4e3-454f-a1dc-d338e0226046 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:04.572119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58216d2a-5825-4ee9-ae7c-03887c091341 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:04.602697 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:04.612413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f39d6-9d36-4a51-96aa-1cdc216be1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:04.613246 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f39d6-9d36-4a51-96aa-1cdc216be1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:04.613582 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f39d6-9d36-4a51-96aa-1cdc216be1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:04.614532 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f0f39d6-9d36-4a51-96aa-1cdc216be1a8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:04.625718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f0f39d6-9d36-4a51-96aa-1cdc216be1a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:04.634510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-7f0f39d6-9d36-4a51-96aa-1cdc216be1a8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:53:04.634510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:04.634510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:04.634510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:04.634510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:04.634510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:04.634510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:04.634510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:04.634510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:04.634510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:04.634510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:04.634510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:04.634510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:04.634510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:04.634510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:04.634510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:04.635418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:04.635418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:04.635418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:04.635418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:04.635418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:04.635418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:04.635418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:04.635418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:04.635418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:04.635418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:04.635418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:04.635418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:04.635418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:04.635418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:04.635418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:04.635418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:04.641282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:04.641282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:04.641282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:04.641282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:04.641282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:04.641282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:04.641282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:04.641282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:04.641282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:04.641282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:04.641282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:53:04.641282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:04.641282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 121/228] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:53:04 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:04.641282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f39d6-9d36-4a51-96aa-1cdc216be1a8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:04.654204 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca1667c3-f4e3-454f-a1dc-d338e0226046 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:04.692469 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0d9ea-6de4-4c28-a68c-f2c02a072a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:04.693520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0d9ea-6de4-4c28-a68c-f2c02a072a58 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:04.694023 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0d9ea-6de4-4c28-a68c-f2c02a072a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:04.694528 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b0d9ea-6de4-4c28-a68c-f2c02a072a58 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:04.695570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b0d9ea-6de4-4c28-a68c-f2c02a072a58 None admin] RBAC: Adding query filter params (role.id=827f1f1239754bf797e64f270ba1a80c, scope.project.id=34304bd3ae654c1c976f55ca788ae592, user.id=05bf542db18b415087bef3ca1a0395c8) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:04.696151 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b0d9ea-6de4-4c28-a68c-f2c02a072a58 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:04.697952 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23b0d9ea-6de4-4c28-a68c-f2c02a072a58 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:04.704358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 108/229] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:53:04 2022] GET /identity/v3/role_assignments?user.id=05bf542db18b415087bef3ca1a0395c8&scope.project.id=34304bd3ae654c1c976f55ca788ae592&role.id=827f1f1239754bf797e64f270ba1a80c => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:04.705597 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0d9ea-6de4-4c28-a68c-f2c02a072a58 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:04.715906 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:04.753803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f0f39d6-9d36-4a51-96aa-1cdc216be1a8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:04.792583 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31de95d-8a7d-45c7-8913-03aec262fc24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:04.793633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31de95d-8a7d-45c7-8913-03aec262fc24 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:04.793871 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31de95d-8a7d-45c7-8913-03aec262fc24 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:04.794153 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a31de95d-8a7d-45c7-8913-03aec262fc24 None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592/users/d0ccab2bdaae4f7ba089039a9f064a7b/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:04.796375 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31de95d-8a7d-45c7-8913-03aec262fc24 None admin] RBAC: Authorizing `identity:create_grant(project_id=34304bd3ae654c1c976f55ca788ae592, user_id=d0ccab2bdaae4f7ba089039a9f064a7b, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:04.798598 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a31de95d-8a7d-45c7-8913-03aec262fc24 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:04.816884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 122/230] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:53:04 2022] PUT /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592/users/d0ccab2bdaae4f7ba089039a9f064a7b/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:53:04.825817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b0d9ea-6de4-4c28-a68c-f2c02a072a58 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:04.860231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:04.870895 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232e62a-64bb-417f-94b8-930737cc076b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:04.872024 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232e62a-64bb-417f-94b8-930737cc076b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:04.872569 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232e62a-64bb-417f-94b8-930737cc076b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:04.873063 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1232e62a-64bb-417f-94b8-930737cc076b None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:04.874092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232e62a-64bb-417f-94b8-930737cc076b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:04.874660 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232e62a-64bb-417f-94b8-930737cc076b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:04.876426 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1232e62a-64bb-417f-94b8-930737cc076b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:04.887250 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 109/231] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:04 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:04.893893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31de95d-8a7d-45c7-8913-03aec262fc24 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:05.054392 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a31de95d-8a7d-45c7-8913-03aec262fc24 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:05.095789 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:05.101486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa94740-089d-4235-ab27-225d9e399f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:05.103105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa94740-089d-4235-ab27-225d9e399f87 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:05.103790 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa94740-089d-4235-ab27-225d9e399f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:05.104689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa94740-089d-4235-ab27-225d9e399f87 None admin] PATH_INFO: `/v3/users/d0ccab2bdaae4f7ba089039a9f064a7b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:05.106055 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa94740-089d-4235-ab27-225d9e399f87 None admin] RBAC: Authorizing `identity:get_user(user_id=d0ccab2bdaae4f7ba089039a9f064a7b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:05.107664 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa94740-089d-4235-ab27-225d9e399f87 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:05.110099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 123/232] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:53:04 2022] GET /identity/v3/users/d0ccab2bdaae4f7ba089039a9f064a7b => generated 283 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:05.115814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232e62a-64bb-417f-94b8-930737cc076b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:05.235408 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1232e62a-64bb-417f-94b8-930737cc076b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:05.283884 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b6d76b-3962-4b4a-9bd5-c42b60bc467b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:05.285215 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6d76b-3962-4b4a-9bd5-c42b60bc467b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:05.285682 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6d76b-3962-4b4a-9bd5-c42b60bc467b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:05.286135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b6d76b-3962-4b4a-9bd5-c42b60bc467b None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:05.287554 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6d76b-3962-4b4a-9bd5-c42b60bc467b None admin] RBAC: Authorizing `identity:get_project(project_id=34304bd3ae654c1c976f55ca788ae592)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:05.289197 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b6d76b-3962-4b4a-9bd5-c42b60bc467b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:05.291154 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 110/233] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:05 2022] GET /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:05.296871 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa94740-089d-4235-ab27-225d9e399f87 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:05.429805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa94740-089d-4235-ab27-225d9e399f87 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:05.482534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aebe038-f0c4-418d-8c8b-eaed91fc5a89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:05.485330 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebe038-f0c4-418d-8c8b-eaed91fc5a89 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:05.485330 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebe038-f0c4-418d-8c8b-eaed91fc5a89 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:05.485330 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aebe038-f0c4-418d-8c8b-eaed91fc5a89 None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592/users/d0ccab2bdaae4f7ba089039a9f064a7b/roles/37f000302df54045892d35f57104660a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:05.491148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebe038-f0c4-418d-8c8b-eaed91fc5a89 None admin] RBAC: Authorizing `identity:create_grant(project_id=34304bd3ae654c1c976f55ca788ae592, user_id=d0ccab2bdaae4f7ba089039a9f064a7b, role_id=37f000302df54045892d35f57104660a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:05.492816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aebe038-f0c4-418d-8c8b-eaed91fc5a89 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:05.515086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 124/234] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:53:05 2022] PUT /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592/users/d0ccab2bdaae4f7ba089039a9f064a7b/roles/37f000302df54045892d35f57104660a => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:53:06.299049 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed08dd8-00d4-47f8-bb6c-82f335126b31 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:06.299590 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed08dd8-00d4-47f8-bb6c-82f335126b31 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:06.299590 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed08dd8-00d4-47f8-bb6c-82f335126b31 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:06.301129 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 111/235] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:06.311157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da565b2f-5e39-473c-8b9a-7a0f2478c897 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:06.312035 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da565b2f-5e39-473c-8b9a-7a0f2478c897 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:06.312495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da565b2f-5e39-473c-8b9a-7a0f2478c897 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:06.324334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64045daa-dfca-4661-ab9d-2f51ce972181 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:06.324713 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64045daa-dfca-4661-ab9d-2f51ce972181 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:06.325120 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64045daa-dfca-4661-ab9d-2f51ce972181 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:06.326722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 112/236] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:53:06.336000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7a85d-2417-474e-a046-d00c5cdc6bac None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:06.336294 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7a85d-2417-474e-a046-d00c5cdc6bac None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:06.336578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa7a85d-2417-474e-a046-d00c5cdc6bac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:06.340113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-da565b2f-5e39-473c-8b9a-7a0f2478c897 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:06.341323 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-da565b2f-5e39-473c-8b9a-7a0f2478c897 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:06.364911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1aa7a85d-2417-474e-a046-d00c5cdc6bac None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:06.366107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1aa7a85d-2417-474e-a046-d00c5cdc6bac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:06.443322 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:06.453062 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:06.476024 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 125/237] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:06.483946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 113/238] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:06.499837 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da565b2f-5e39-473c-8b9a-7a0f2478c897 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:06.535653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7a85d-2417-474e-a046-d00c5cdc6bac None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:06.655144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da565b2f-5e39-473c-8b9a-7a0f2478c897 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:06.696023 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa7a85d-2417-474e-a046-d00c5cdc6bac None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:06.701595 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5be049-804b-42ea-b555-79745337db69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:06.702474 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5be049-804b-42ea-b555-79745337db69 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:06.702770 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5be049-804b-42ea-b555-79745337db69 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:06.703049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef5be049-804b-42ea-b555-79745337db69 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:06.726156 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef5be049-804b-42ea-b555-79745337db69 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:06.734125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-ef5be049-804b-42ea-b555-79745337db69 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 13:53:06.734125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:06.734125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:06.734125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:06.734125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:06.734125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:06.734125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:06.734125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:06.734125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:06.734125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:06.734125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:06.734125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:06.734125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:06.734125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:06.734125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:06.734125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:06.735042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:06.735042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:06.735042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:06.735042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:06.735042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:06.735042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:06.735042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:06.735042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:06.735042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:06.735042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:06.735042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:06.735042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:06.735042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:06.735042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:06.735042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:06.735042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:06.735818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:06.735818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:06.735818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:06.735818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:06.735818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:06.735818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:06.735818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:06.735818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:06.735818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:06.735818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:06.735818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 13:53:06.735818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:06.735818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 126/239] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:06.742501 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5be049-804b-42ea-b555-79745337db69 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:06.778323 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849912b9-d233-44b9-975e-0b4f7c669e47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:06.779148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849912b9-d233-44b9-975e-0b4f7c669e47 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:06.779438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849912b9-d233-44b9-975e-0b4f7c669e47 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:06.779715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-849912b9-d233-44b9-975e-0b4f7c669e47 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:06.783045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849912b9-d233-44b9-975e-0b4f7c669e47 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:06.784894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-849912b9-d233-44b9-975e-0b4f7c669e47 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:06.788279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 114/240] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 253 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:06.796093 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849912b9-d233-44b9-975e-0b4f7c669e47 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:06.859134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:06.863736 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:06.913945 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-849912b9-d233-44b9-975e-0b4f7c669e47 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:06.951126 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de2ccb-61eb-437f-b72b-5fca696f5672 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:06.952178 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de2ccb-61eb-437f-b72b-5fca696f5672 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:06.952664 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de2ccb-61eb-437f-b72b-5fca696f5672 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:06.953155 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75de2ccb-61eb-437f-b72b-5fca696f5672 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:06.954712 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de2ccb-61eb-437f-b72b-5fca696f5672 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:06.956446 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75de2ccb-61eb-437f-b72b-5fca696f5672 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:06.983429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 115/241] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 11 13:53:06 2022] POST /identity/v3/users => generated 293 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:53:06.991669 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef5be049-804b-42ea-b555-79745337db69 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:07.029921 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af7fad-673d-472f-a1d8-e41a40ef6285 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:07.031008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af7fad-673d-472f-a1d8-e41a40ef6285 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:07.031592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af7fad-673d-472f-a1d8-e41a40ef6285 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:07.032143 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58af7fad-673d-472f-a1d8-e41a40ef6285 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:07.033153 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af7fad-673d-472f-a1d8-e41a40ef6285 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:07.033678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af7fad-673d-472f-a1d8-e41a40ef6285 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:07.035369 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58af7fad-673d-472f-a1d8-e41a40ef6285 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:07.053957 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 127/242] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:53:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 312 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:07.060128 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de2ccb-61eb-437f-b72b-5fca696f5672 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:07.209187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75de2ccb-61eb-437f-b72b-5fca696f5672 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:07.221059 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:07.251432 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-134d3119-da6b-428d-b08f-6d3bad12ea03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:07.252746 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d3119-da6b-428d-b08f-6d3bad12ea03 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:07.253230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d3119-da6b-428d-b08f-6d3bad12ea03 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:07.253702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-134d3119-da6b-428d-b08f-6d3bad12ea03 None admin] PATH_INFO: `/v3/users/e9d06cdd6d164fe8a90b30959493e066` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:07.268080 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d3119-da6b-428d-b08f-6d3bad12ea03 None admin] RBAC: Authorizing `identity:get_user(user_id=e9d06cdd6d164fe8a90b30959493e066)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:07.269855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-134d3119-da6b-428d-b08f-6d3bad12ea03 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:07.272167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 116/243] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:53:07 2022] GET /identity/v3/users/e9d06cdd6d164fe8a90b30959493e066 => generated 297 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:07.277790 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af7fad-673d-472f-a1d8-e41a40ef6285 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:07.387862 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58af7fad-673d-472f-a1d8-e41a40ef6285 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:07.402926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:07.428710 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ebfd54-532e-4866-a0df-b75e9ce77284 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:07.429379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebfd54-532e-4866-a0df-b75e9ce77284 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:07.429666 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebfd54-532e-4866-a0df-b75e9ce77284 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:07.430190 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ebfd54-532e-4866-a0df-b75e9ce77284 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:07.435296 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499fa36-42bb-4348-b48b-4d43fca87b51 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:07.435777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499fa36-42bb-4348-b48b-4d43fca87b51 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:07.435864 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3499fa36-42bb-4348-b48b-4d43fca87b51 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:07.437235 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 117/244] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:53:07.442632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d58c5-f5a8-4dc0-ad05-258e4023bfc1 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:07.442925 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d58c5-f5a8-4dc0-ad05-258e4023bfc1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:07.443202 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab6d58c5-f5a8-4dc0-ad05-258e4023bfc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:07.461554 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ab6d58c5-f5a8-4dc0-ad05-258e4023bfc1 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:07.462840 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ab6d58c5-f5a8-4dc0-ad05-258e4023bfc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:07.533239 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ebfd54-532e-4866-a0df-b75e9ce77284 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:07.534769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ebfd54-532e-4866-a0df-b75e9ce77284 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:07.557527 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 118/245] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:53:07.582379 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d58c5-f5a8-4dc0-ad05-258e4023bfc1 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:07.669524 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 128/246] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:07.670910 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d57f54-0589-4944-90d8-531e328d738a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:07.671203 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d57f54-0589-4944-90d8-531e328d738a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:07.671484 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d57f54-0589-4944-90d8-531e328d738a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:07.673746 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 129/247] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:07.682860 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1ecb7-26a7-48ec-bf8b-fd82e735b615 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:07.683149 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1ecb7-26a7-48ec-bf8b-fd82e735b615 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:07.683433 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae1ecb7-26a7-48ec-bf8b-fd82e735b615 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:07.703522 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cae1ecb7-26a7-48ec-bf8b-fd82e735b615 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:07.704740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cae1ecb7-26a7-48ec-bf8b-fd82e735b615 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:07.706462 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:07.712957 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab6d58c5-f5a8-4dc0-ad05-258e4023bfc1 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:07.717576 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:07.759318 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddf549-27e6-4ace-800b-493e72447e96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:07.760138 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddf549-27e6-4ace-800b-493e72447e96 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:07.760422 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddf549-27e6-4ace-800b-493e72447e96 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:07.760702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ddf549-27e6-4ace-800b-493e72447e96 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:07.772591 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ddf549-27e6-4ace-800b-493e72447e96 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:07.781100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-d8ddf549-27e6-4ace-800b-493e72447e96 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:07.781100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:07.781100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:07.781100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:07.781100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:07.781100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:07.781100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:07.781100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:07.781100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:07.781100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:07.781100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:07.781100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:07.781100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:07.781100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:07.781100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:07.781100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:07.782619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:07.782619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:07.782619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:07.782619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:07.782619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:07.782619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:07.782619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:07.782619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:07.782619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:07.782619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:07.782619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:07.782619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:07.782619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:07.782619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:07.782619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:07.782619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:07.783410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:07.783410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:07.783410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:07.783410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:07.783410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:07.783410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:07.783410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:07.783410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:07.783410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:07.783410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:07.783410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:07.783410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:07.783410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 119/248] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:07 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:07.784787 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddf549-27e6-4ace-800b-493e72447e96 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:07.787806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 130/249] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:53:07.791476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1ecb7-26a7-48ec-bf8b-fd82e735b615 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:07.900861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae1ecb7-26a7-48ec-bf8b-fd82e735b615 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:07.938379 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ddf549-27e6-4ace-800b-493e72447e96 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:07.940092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0beeea2-cefd-4a36-8f5c-63d765751e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:07.940771 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0beeea2-cefd-4a36-8f5c-63d765751e4f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:07.941053 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0beeea2-cefd-4a36-8f5c-63d765751e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:07.941341 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0beeea2-cefd-4a36-8f5c-63d765751e4f None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:07.942100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0beeea2-cefd-4a36-8f5c-63d765751e4f None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:07.942439 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0beeea2-cefd-4a36-8f5c-63d765751e4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:07.943816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0beeea2-cefd-4a36-8f5c-63d765751e4f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:07.959353 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 131/250] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:53:07 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:07.959353 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0beeea2-cefd-4a36-8f5c-63d765751e4f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:07.964982 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:07.987115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea12692-50ab-4795-83cf-0c69ca919402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:07.988443 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea12692-50ab-4795-83cf-0c69ca919402 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:07.988724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea12692-50ab-4795-83cf-0c69ca919402 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:07.989052 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea12692-50ab-4795-83cf-0c69ca919402 None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:07.990293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea12692-50ab-4795-83cf-0c69ca919402 None admin] RBAC: Authorizing `identity:get_project(project_id=34304bd3ae654c1c976f55ca788ae592)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:07.991779 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea12692-50ab-4795-83cf-0c69ca919402 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:07.994510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 120/251] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:07 2022] GET /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592 => generated 302 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:07.996015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87502353-3edd-425a-8c73-dfc1ab073579 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:07.996709 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87502353-3edd-425a-8c73-dfc1ab073579 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:07.997027 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87502353-3edd-425a-8c73-dfc1ab073579 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:07.999093 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 121/252] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:08.003120 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87502353-3edd-425a-8c73-dfc1ab073579 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:08.043663 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:08.103427 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0beeea2-cefd-4a36-8f5c-63d765751e4f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:08.141531 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cee396-1fa3-4d19-9369-fa4a3f421d74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:08.142414 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cee396-1fa3-4d19-9369-fa4a3f421d74 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:08.142701 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cee396-1fa3-4d19-9369-fa4a3f421d74 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:08.142974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cee396-1fa3-4d19-9369-fa4a3f421d74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:08.143474 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87502353-3edd-425a-8c73-dfc1ab073579 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:08.156445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cee396-1fa3-4d19-9369-fa4a3f421d74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:08.167419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-32cee396-1fa3-4d19-9369-fa4a3f421d74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:53:08.167419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:08.167419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:08.167419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:08.167419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:08.167419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:08.167419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:08.167419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:08.167419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:08.167419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:08.167419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:08.167419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:08.167419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:08.167419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:08.167419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:08.167419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:08.168304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:08.168304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:08.168304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:08.168304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:08.168304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:08.168304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:08.168304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:08.168304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:08.168304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:08.168304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:08.168304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:08.168304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:08.168304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:08.168304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:08.168304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:08.168304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:08.169070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:08.169070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:08.169070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:08.169070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:08.169070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:08.169070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:08.169070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:08.169070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:08.169070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:08.169070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:08.169070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:53:08.169070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:08.171907 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 132/253] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:53:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:08.173233 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cee396-1fa3-4d19-9369-fa4a3f421d74 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:08.185643 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d06355b-91f8-463c-9425-02ad0991561a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:08.186712 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06355b-91f8-463c-9425-02ad0991561a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:08.187098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06355b-91f8-463c-9425-02ad0991561a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:08.187477 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d06355b-91f8-463c-9425-02ad0991561a None admin] PATH_INFO: `/v3/users/05bf542db18b415087bef3ca1a0395c8` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:08.189447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06355b-91f8-463c-9425-02ad0991561a None admin] RBAC: Authorizing `identity:get_user(user_id=05bf542db18b415087bef3ca1a0395c8)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:08.191143 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d06355b-91f8-463c-9425-02ad0991561a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:08.193148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 122/254] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:53:07 2022] GET /identity/v3/users/05bf542db18b415087bef3ca1a0395c8 => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:08.198852 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169045a-ad93-4394-87e9-1e7dde53be3a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:08.199217 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169045a-ad93-4394-87e9-1e7dde53be3a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:08.199547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f169045a-ad93-4394-87e9-1e7dde53be3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:08.217809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-f169045a-ad93-4394-87e9-1e7dde53be3a None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:08.219098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f169045a-ad93-4394-87e9-1e7dde53be3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:08.260068 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:08.296168 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 123/255] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:08.303398 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169045a-ad93-4394-87e9-1e7dde53be3a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:08.304347 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cee396-1fa3-4d19-9369-fa4a3f421d74 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:08.337757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:08.364511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d58d62-2cc1-4728-98c8-9f04ccab1625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:08.365235 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d58d62-2cc1-4728-98c8-9f04ccab1625 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:08.365531 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d58d62-2cc1-4728-98c8-9f04ccab1625 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:08.365817 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d58d62-2cc1-4728-98c8-9f04ccab1625 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:08.366814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d58d62-2cc1-4728-98c8-9f04ccab1625 None admin] RBAC: Adding query filter params (role.id=3a110b9d612545b09bf08554a2dd7eac, scope.project.id=34304bd3ae654c1c976f55ca788ae592, user.id=e9d06cdd6d164fe8a90b30959493e066) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:08.367135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d58d62-2cc1-4728-98c8-9f04ccab1625 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:08.368592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d58d62-2cc1-4728-98c8-9f04ccab1625 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:08.375815 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 133/256] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:53:08 2022] GET /identity/v3/role_assignments?user.id=e9d06cdd6d164fe8a90b30959493e066&scope.project.id=34304bd3ae654c1c976f55ca788ae592&role.id=3a110b9d612545b09bf08554a2dd7eac => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:08.376443 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d58d62-2cc1-4728-98c8-9f04ccab1625 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:08.421937 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f169045a-ad93-4394-87e9-1e7dde53be3a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:08.460709 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2d3ed-c232-4063-8a20-b841bf375e01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:08.462273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2d3ed-c232-4063-8a20-b841bf375e01 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:08.462797 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2d3ed-c232-4063-8a20-b841bf375e01 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:08.463270 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf2d3ed-c232-4063-8a20-b841bf375e01 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:08.464265 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf2d3ed-c232-4063-8a20-b841bf375e01 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:08.464771 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf2d3ed-c232-4063-8a20-b841bf375e01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:08.466388 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf2d3ed-c232-4063-8a20-b841bf375e01 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:08.484403 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 124/257] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:08 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:08.485603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2d3ed-c232-4063-8a20-b841bf375e01 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:08.495471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d58d62-2cc1-4728-98c8-9f04ccab1625 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:08.526099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:08.536536 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41210a-2316-4603-a897-2d02d58254f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:08.537710 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41210a-2316-4603-a897-2d02d58254f9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:08.538252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41210a-2316-4603-a897-2d02d58254f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:08.538869 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec41210a-2316-4603-a897-2d02d58254f9 None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:08.540318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41210a-2316-4603-a897-2d02d58254f9 None admin] RBAC: Authorizing `identity:get_project(project_id=34304bd3ae654c1c976f55ca788ae592)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:08.542331 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec41210a-2316-4603-a897-2d02d58254f9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:08.544815 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 134/258] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:08 2022] GET /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592 => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:08.545940 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41210a-2316-4603-a897-2d02d58254f9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:08.617998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:08.637625 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf2d3ed-c232-4063-8a20-b841bf375e01 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:08.680547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41f7f7-6cd3-4332-bb3a-888dc10f90a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:08.682433 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41f7f7-6cd3-4332-bb3a-888dc10f90a0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:08.683228 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41f7f7-6cd3-4332-bb3a-888dc10f90a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:08.684893 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41f7f7-6cd3-4332-bb3a-888dc10f90a0 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:08.702939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41f7f7-6cd3-4332-bb3a-888dc10f90a0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:08.712028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-0e41f7f7-6cd3-4332-bb3a-888dc10f90a0 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:53:08.712028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:08.712028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:08.712028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:08.712028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:08.712028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:08.712028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:08.712028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:08.712028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:08.712028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:08.712028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:08.712028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:08.712028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:08.712028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:08.712028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:08.712028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:08.712967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:08.712967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:08.712967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:08.712967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:08.712967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:08.712967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:08.712967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:08.712967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:08.712967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:08.712967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:08.712967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:08.712967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:08.712967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:08.712967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:08.712967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:08.712967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:08.713778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:08.713778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:08.713778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:08.713778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:08.713778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:08.713778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:08.713778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:08.713778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:08.713778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:08.713778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:08.713778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:53:08.713778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:08.715875 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 125/259] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:53:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:08.716842 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41f7f7-6cd3-4332-bb3a-888dc10f90a0 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:08.722116 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec41210a-2316-4603-a897-2d02d58254f9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:08.761168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9805c91-5104-4ad2-9777-914378c50458 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:08.762226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9805c91-5104-4ad2-9777-914378c50458 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:08.762567 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9805c91-5104-4ad2-9777-914378c50458 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:08.762874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9805c91-5104-4ad2-9777-914378c50458 None admin] PATH_INFO: `/v3/users/d0ccab2bdaae4f7ba089039a9f064a7b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:08.765915 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9805c91-5104-4ad2-9777-914378c50458 None admin] RBAC: Authorizing `identity:get_user(user_id=d0ccab2bdaae4f7ba089039a9f064a7b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:08.767609 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9805c91-5104-4ad2-9777-914378c50458 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:08.769415 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 135/260] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:53:08 2022] GET /identity/v3/users/d0ccab2bdaae4f7ba089039a9f064a7b => generated 283 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:08.770200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9805c91-5104-4ad2-9777-914378c50458 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:08.862810 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:08.896914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9805c91-5104-4ad2-9777-914378c50458 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:08.929935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41f7f7-6cd3-4332-bb3a-888dc10f90a0 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:08.940886 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce5b61-4b2e-4345-89c8-f74492e74ff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:08.941576 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce5b61-4b2e-4345-89c8-f74492e74ff4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:08.941870 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce5b61-4b2e-4345-89c8-f74492e74ff4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:08.942148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce5b61-4b2e-4345-89c8-f74492e74ff4 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:08.947255 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce5b61-4b2e-4345-89c8-f74492e74ff4 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:08.947578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce5b61-4b2e-4345-89c8-f74492e74ff4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:08.948996 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce5b61-4b2e-4345-89c8-f74492e74ff4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:08.961986 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 136/261] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:08.962685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce5b61-4b2e-4345-89c8-f74492e74ff4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:09.000029 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6264fc-1755-4ac3-856c-1987901a82d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:09.002556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6264fc-1755-4ac3-856c-1987901a82d6 None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:09.004303 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6264fc-1755-4ac3-856c-1987901a82d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:09.005454 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6264fc-1755-4ac3-856c-1987901a82d6 None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592/users/05bf542db18b415087bef3ca1a0395c8/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:09.009611 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6264fc-1755-4ac3-856c-1987901a82d6 None admin] RBAC: Authorizing `identity:create_grant(project_id=34304bd3ae654c1c976f55ca788ae592, user_id=05bf542db18b415087bef3ca1a0395c8, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:09.012685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6264fc-1755-4ac3-856c-1987901a82d6 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:09.039866 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 126/262] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:53:08 2022] PUT /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592/users/05bf542db18b415087bef3ca1a0395c8/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:53:09.041088 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6264fc-1755-4ac3-856c-1987901a82d6 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:09.063834 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:09.181016 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce5b61-4b2e-4345-89c8-f74492e74ff4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:09.235288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:09.251557 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6264fc-1755-4ac3-856c-1987901a82d6 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:09.253490 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee5c8d-53c2-4638-9b94-84b1f49ef3af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:09.254296 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee5c8d-53c2-4638-9b94-84b1f49ef3af None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:09.255421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee5c8d-53c2-4638-9b94-84b1f49ef3af None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:09.255421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ee5c8d-53c2-4638-9b94-84b1f49ef3af None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:09.294126 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d2269-ed2e-497d-ba9b-e2085a00767b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:09.295884 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d2269-ed2e-497d-ba9b-e2085a00767b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:09.296174 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d2269-ed2e-497d-ba9b-e2085a00767b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:09.296453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314d2269-ed2e-497d-ba9b-e2085a00767b None admin] PATH_INFO: `/v3/users/d0ccab2bdaae4f7ba089039a9f064a7b` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:09.301543 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d2269-ed2e-497d-ba9b-e2085a00767b None admin] RBAC: Authorizing `identity:get_user(user_id=d0ccab2bdaae4f7ba089039a9f064a7b)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:09.303077 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314d2269-ed2e-497d-ba9b-e2085a00767b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:09.306211 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 127/263] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:53:09 2022] GET /identity/v3/users/d0ccab2bdaae4f7ba089039a9f064a7b => generated 283 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:09.313297 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d2269-ed2e-497d-ba9b-e2085a00767b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:09.394356 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee5c8d-53c2-4638-9b94-84b1f49ef3af None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:09.395860 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ee5c8d-53c2-4638-9b94-84b1f49ef3af None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:09.458069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:09.508864 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314d2269-ed2e-497d-ba9b-e2085a00767b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:09.548019 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 137/264] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 586 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:09.553691 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee5c8d-53c2-4638-9b94-84b1f49ef3af None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:09.597963 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:09.624065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ad182-5493-40e4-8c1e-7373fefcfc29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:09.624750 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ad182-5493-40e4-8c1e-7373fefcfc29 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:09.625040 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ad182-5493-40e4-8c1e-7373fefcfc29 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:09.625365 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce6ad182-5493-40e4-8c1e-7373fefcfc29 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:09.713724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ee5c8d-53c2-4638-9b94-84b1f49ef3af None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:09.757453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ad182-5493-40e4-8c1e-7373fefcfc29 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:09.760981 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce6ad182-5493-40e4-8c1e-7373fefcfc29 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:09.781108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879fc40-ff13-4234-9097-9ccc801d5025 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:09.782755 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879fc40-ff13-4234-9097-9ccc801d5025 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:09.783104 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879fc40-ff13-4234-9097-9ccc801d5025 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:09.783405 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c879fc40-ff13-4234-9097-9ccc801d5025 None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:09.785307 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c879fc40-ff13-4234-9097-9ccc801d5025 None admin] RBAC: Authorizing `identity:get_project(project_id=34304bd3ae654c1c976f55ca788ae592)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:09.787295 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c879fc40-ff13-4234-9097-9ccc801d5025 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:09.789748 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 138/265] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:09 2022] GET /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592 => generated 302 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:09.798269 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879fc40-ff13-4234-9097-9ccc801d5025 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:09.907861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:09.960054 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 128/266] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 647 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:09.966242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ad182-5493-40e4-8c1e-7373fefcfc29 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:10.006037 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:10.014420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c879fc40-ff13-4234-9097-9ccc801d5025 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:10.060495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36fdecb8-a439-490a-9ae5-80bf07fc14fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:10.061084 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdecb8-a439-490a-9ae5-80bf07fc14fd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:10.061377 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdecb8-a439-490a-9ae5-80bf07fc14fd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:10.061657 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36fdecb8-a439-490a-9ae5-80bf07fc14fd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:10.062773 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdecb8-a439-490a-9ae5-80bf07fc14fd None admin] RBAC: Adding query filter params (role.id=6964639a7f6c41e8a6701ef533151c41, scope.project.id=34304bd3ae654c1c976f55ca788ae592, user.id=d0ccab2bdaae4f7ba089039a9f064a7b) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:10.063095 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdecb8-a439-490a-9ae5-80bf07fc14fd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:10.064549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36fdecb8-a439-490a-9ae5-80bf07fc14fd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:10.077390 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 139/267] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:53:09 2022] GET /identity/v3/role_assignments?user.id=d0ccab2bdaae4f7ba089039a9f064a7b&scope.project.id=34304bd3ae654c1c976f55ca788ae592&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 608 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:10.079310 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881176e9-d047-4a66-9d8c-0206cc112ebe None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:10.079608 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881176e9-d047-4a66-9d8c-0206cc112ebe None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:10.079893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881176e9-d047-4a66-9d8c-0206cc112ebe None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:10.081634 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 140/268] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:10.087663 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a8555-cc74-4b3f-975c-fea79a34836f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:10.087965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a8555-cc74-4b3f-975c-fea79a34836f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:10.088234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf1a8555-cc74-4b3f-975c-fea79a34836f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:10.108134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cf1a8555-cc74-4b3f-975c-fea79a34836f None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:10.109409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cf1a8555-cc74-4b3f-975c-fea79a34836f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:10.171786 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce6ad182-5493-40e4-8c1e-7373fefcfc29 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:10.201355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 141/269] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:10.207329 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4f56c-7eb9-43e6-bf61-a7ab907be6aa None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:10.208159 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4f56c-7eb9-43e6-bf61-a7ab907be6aa None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:10.208479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea4f56c-7eb9-43e6-bf61-a7ab907be6aa None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:10.209999 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 142/270] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:10 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:10.217192 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6c56e-c8db-4efb-8d4e-da258f8f4bce None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:10.217587 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6c56e-c8db-4efb-8d4e-da258f8f4bce None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:10.217780 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e6c56e-c8db-4efb-8d4e-da258f8f4bce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:10.245784 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-01e6c56e-c8db-4efb-8d4e-da258f8f4bce None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:10.247591 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-01e6c56e-c8db-4efb-8d4e-da258f8f4bce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:10.274704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60de7f-c839-4bfd-be7c-29f547eaaa8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:10.275973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60de7f-c839-4bfd-be7c-29f547eaaa8a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:10.276473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60de7f-c839-4bfd-be7c-29f547eaaa8a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:10.276949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd60de7f-c839-4bfd-be7c-29f547eaaa8a None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:10.278449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd60de7f-c839-4bfd-be7c-29f547eaaa8a None admin] RBAC: Authorizing `identity:get_project(project_id=34304bd3ae654c1c976f55ca788ae592)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:10.280157 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd60de7f-c839-4bfd-be7c-29f547eaaa8a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:10.282814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 129/271] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:09 2022] GET /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592 => generated 302 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:10.283810 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60de7f-c839-4bfd-be7c-29f547eaaa8a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:10.353885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:10.356450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 143/272] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:53:10.357460 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6c56e-c8db-4efb-8d4e-da258f8f4bce None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:10.382796 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:10.439450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd60de7f-c839-4bfd-be7c-29f547eaaa8a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:10.491297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e6c56e-c8db-4efb-8d4e-da258f8f4bce None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:10.491909 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7601ce-d1dc-478d-bf18-7d040a9b8baa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:10.492730 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7601ce-d1dc-478d-bf18-7d040a9b8baa None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:10.493018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7601ce-d1dc-478d-bf18-7d040a9b8baa None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:10.493297 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7601ce-d1dc-478d-bf18-7d040a9b8baa None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:10.508813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7601ce-d1dc-478d-bf18-7d040a9b8baa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:10.521894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-3d7601ce-d1dc-478d-bf18-7d040a9b8baa None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:10.521894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:10.521894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:10.521894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:10.521894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:10.521894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:10.521894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:10.521894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:10.521894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:10.521894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:10.521894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:10.521894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:10.521894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:10.521894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:10.521894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:10.521894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:10.523268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:10.523268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:10.523268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:10.523268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:10.523268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:10.523268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:10.523268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:10.523268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:10.523268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:10.523268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:10.523268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:10.523268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:10.523268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:10.523268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:10.523268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:10.523268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:10.524046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:10.524046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:10.524046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:10.524046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:10.524046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:10.524046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:10.524046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:10.524046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:10.524046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:10.524046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:10.524046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:10.524046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:10.524046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 130/273] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:10.524046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7601ce-d1dc-478d-bf18-7d040a9b8baa None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:10.543027 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd69ec-bed5-4800-af05-61971aaeaf10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:10.546496 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd69ec-bed5-4800-af05-61971aaeaf10 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:10.546797 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd69ec-bed5-4800-af05-61971aaeaf10 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:10.547125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71dd69ec-bed5-4800-af05-61971aaeaf10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:10.547996 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd69ec-bed5-4800-af05-61971aaeaf10 None admin] RBAC: Adding query filter params (role.id=37f000302df54045892d35f57104660a, scope.project.id=34304bd3ae654c1c976f55ca788ae592, user.id=d0ccab2bdaae4f7ba089039a9f064a7b) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:10.548319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd69ec-bed5-4800-af05-61971aaeaf10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:10.549787 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71dd69ec-bed5-4800-af05-61971aaeaf10 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:10.560244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 144/274] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:53:10 2022] GET /identity/v3/role_assignments?user.id=d0ccab2bdaae4f7ba089039a9f064a7b&scope.project.id=34304bd3ae654c1c976f55ca788ae592&role.id=37f000302df54045892d35f57104660a => generated 608 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:10.561075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd69ec-bed5-4800-af05-61971aaeaf10 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:10.669795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:10.677322 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7601ce-d1dc-478d-bf18-7d040a9b8baa None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:10.680728 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:10.693023 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71dd69ec-bed5-4800-af05-61971aaeaf10 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:10.728576 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8bdc83-1bcf-40cf-9d8e-cba99028a4a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:10.730639 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8bdc83-1bcf-40cf-9d8e-cba99028a4a0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:10.731277 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8bdc83-1bcf-40cf-9d8e-cba99028a4a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:10.731848 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a8bdc83-1bcf-40cf-9d8e-cba99028a4a0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:10.733594 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3b3ba-318e-4cf5-8885-d774763ca1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:10.733811 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8bdc83-1bcf-40cf-9d8e-cba99028a4a0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:10.734472 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3b3ba-318e-4cf5-8885-d774763ca1f0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:10.734903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3b3ba-318e-4cf5-8885-d774763ca1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:10.735262 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f3b3ba-318e-4cf5-8885-d774763ca1f0 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:10.736166 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3b3ba-318e-4cf5-8885-d774763ca1f0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:10.736512 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a8bdc83-1bcf-40cf-9d8e-cba99028a4a0 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:10.736632 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3b3ba-318e-4cf5-8885-d774763ca1f0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:10.738082 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f3b3ba-318e-4cf5-8885-d774763ca1f0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:10.738922 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 131/275] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:10.750689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8bdc83-1bcf-40cf-9d8e-cba99028a4a0 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:10.763372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 145/276] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:53:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:10.769868 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3b3ba-318e-4cf5-8885-d774763ca1f0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:10.902290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a8bdc83-1bcf-40cf-9d8e-cba99028a4a0 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:10.908447 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f3b3ba-318e-4cf5-8885-d774763ca1f0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:10.939171 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee5209-5f31-4ba8-bbd6-464df433dc84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:10.939984 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee5209-5f31-4ba8-bbd6-464df433dc84 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:10.940385 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee5209-5f31-4ba8-bbd6-464df433dc84 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:10.940673 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baee5209-5f31-4ba8-bbd6-464df433dc84 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:10.941524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee5209-5f31-4ba8-bbd6-464df433dc84 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:10.943142 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baee5209-5f31-4ba8-bbd6-464df433dc84 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:10.946833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68ca4c95-f3ae-4711-a289-519838f927ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:10.947717 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca4c95-f3ae-4711-a289-519838f927ee None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:10.948002 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca4c95-f3ae-4711-a289-519838f927ee None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:10.948278 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68ca4c95-f3ae-4711-a289-519838f927ee None admin] PATH_INFO: `/v3/users/b39c7b757ef94efab0737d70ad55f5f5` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:10.964545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca4c95-f3ae-4711-a289-519838f927ee None admin] RBAC: Authorizing `identity:get_user(user_id=b39c7b757ef94efab0737d70ad55f5f5)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:10.966012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68ca4c95-f3ae-4711-a289-519838f927ee None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:10.967259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 132/277] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 11 13:53:10 2022] POST /identity/v3/users => generated 293 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:53:10.968133 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 146/278] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:53:10 2022] GET /identity/v3/users/b39c7b757ef94efab0737d70ad55f5f5 => generated 293 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:10.973784 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee5209-5f31-4ba8-bbd6-464df433dc84 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:11.008188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:11.092816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baee5209-5f31-4ba8-bbd6-464df433dc84 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:11.133560 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cabd5f-8ba7-41c3-a951-c1e8443a0047 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:11.134332 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cabd5f-8ba7-41c3-a951-c1e8443a0047 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:11.134761 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cabd5f-8ba7-41c3-a951-c1e8443a0047 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:11.135114 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cabd5f-8ba7-41c3-a951-c1e8443a0047 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:11.136073 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cabd5f-8ba7-41c3-a951-c1e8443a0047 None admin] RBAC: Adding query filter params (role.id=827f1f1239754bf797e64f270ba1a80c, user.id=b39c7b757ef94efab0737d70ad55f5f5, scope.system=all) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:11.136465 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cabd5f-8ba7-41c3-a951-c1e8443a0047 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:11.138028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cabd5f-8ba7-41c3-a951-c1e8443a0047 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:11.144753 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 133/279] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Jul 11 13:53:10 2022] GET /identity/v3/role_assignments?user.id=b39c7b757ef94efab0737d70ad55f5f5&scope.system=all&role.id=827f1f1239754bf797e64f270ba1a80c => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:11.454867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d836952-a43c-4f2a-8787-f31c6cd73acd None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:11.455244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d836952-a43c-4f2a-8787-f31c6cd73acd None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:11.455438 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d836952-a43c-4f2a-8787-f31c6cd73acd None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:11.457108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 147/280] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:11.462237 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5fff9-9907-4fab-9046-730d96f15815 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:11.462555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5fff9-9907-4fab-9046-730d96f15815 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:11.462842 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0b5fff9-9907-4fab-9046-730d96f15815 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:11.483385 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b0b5fff9-9907-4fab-9046-730d96f15815 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:11.485243 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b0b5fff9-9907-4fab-9046-730d96f15815 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:11.567581 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 134/281] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:11.590251 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d836952-a43c-4f2a-8787-f31c6cd73acd None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:11.597078 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:11.716184 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d836952-a43c-4f2a-8787-f31c6cd73acd None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:11.752023 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57de4a4-7fdb-4624-a686-2419efc9145f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:11.752826 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57de4a4-7fdb-4624-a686-2419efc9145f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:11.753109 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57de4a4-7fdb-4624-a686-2419efc9145f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:11.753387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57de4a4-7fdb-4624-a686-2419efc9145f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:11.764699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57de4a4-7fdb-4624-a686-2419efc9145f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:11.773181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b57de4a4-7fdb-4624-a686-2419efc9145f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 13:53:11.773181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:11.773181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:11.773181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:11.773181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:11.773181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:11.773181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:11.773181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:11.773181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:11.773181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:11.773181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:11.773181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:11.773181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:11.773181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:11.773181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:11.773181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:11.774042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:11.774042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:11.774042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:11.774042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:11.774042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:11.774042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:11.774042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:11.774042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:11.774042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:11.774042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:11.774042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:11.774042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:11.774042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:11.774042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:11.774042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:11.774042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:11.774856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:11.774856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:11.774856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:11.774856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:11.774856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:11.774856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:11.774856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:11.774856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:11.774856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:11.774856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:11.774856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 13:53:11.774856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:11.774856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 148/282] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:11.782365 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5fff9-9907-4fab-9046-730d96f15815 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:11.821812 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:11.907823 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce677d0-584f-4c34-ae1c-9177a80655ac None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:11.908142 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce677d0-584f-4c34-ae1c-9177a80655ac None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:11.908390 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ce677d0-584f-4c34-ae1c-9177a80655ac None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:11.909904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 149/283] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:11.915363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b98a9-edf5-4701-be08-70fdf3d28b07 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:11.915660 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b98a9-edf5-4701-be08-70fdf3d28b07 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:11.915945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b98a9-edf5-4701-be08-70fdf3d28b07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:11.933201 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f83b98a9-edf5-4701-be08-70fdf3d28b07 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:11.934443 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f83b98a9-edf5-4701-be08-70fdf3d28b07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:11.942072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0b5fff9-9907-4fab-9046-730d96f15815 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:11.999437 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:12.005721 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 150/284] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:53:12.015629 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e3cdd-5c89-4de9-84fb-76b60dc9acd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:12.017159 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e3cdd-5c89-4de9-84fb-76b60dc9acd5 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:12.017647 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e3cdd-5c89-4de9-84fb-76b60dc9acd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:12.018166 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a1e3cdd-5c89-4de9-84fb-76b60dc9acd5 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:12.019209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1e3cdd-5c89-4de9-84fb-76b60dc9acd5 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:12.019715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1e3cdd-5c89-4de9-84fb-76b60dc9acd5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:12.021315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a1e3cdd-5c89-4de9-84fb-76b60dc9acd5 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:12.034854 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b98a9-edf5-4701-be08-70fdf3d28b07 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:12.042989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 135/285] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:53:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:12.048588 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e3cdd-5c89-4de9-84fb-76b60dc9acd5 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:12.208309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:12.220206 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b98a9-edf5-4701-be08-70fdf3d28b07 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:12.246821 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a1e3cdd-5c89-4de9-84fb-76b60dc9acd5 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:12.260693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709c4bb-d824-4d27-bf24-090974434d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:12.261474 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709c4bb-d824-4d27-bf24-090974434d2f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:12.261813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709c4bb-d824-4d27-bf24-090974434d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:12.262087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709c4bb-d824-4d27-bf24-090974434d2f None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:12.273321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b709c4bb-d824-4d27-bf24-090974434d2f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:12.282324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b709c4bb-d824-4d27-bf24-090974434d2f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:12.282324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:12.282324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:12.282324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:12.282324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:12.282324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:12.282324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:12.282324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:12.282324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:12.282324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:12.282324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:12.282324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:12.282324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:12.282324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:12.282324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:12.282324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:12.283168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:12.283168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:12.283168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:12.283168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:12.283168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:12.283168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:12.283168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:12.283168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:12.283168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:12.283168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:12.283168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:12.283168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:12.283168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:12.283168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:12.283168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:12.283168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:12.283945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:12.283945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:12.283945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:12.283945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:12.283945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:12.283945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:12.283945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:12.283945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:12.283945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:12.283945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:12.283945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:12.283945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:12.286429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 151/286] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:12 2022] GET /identity/v3/roles/member => generated 84 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:53:12.288577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709c4bb-d824-4d27-bf24-090974434d2f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:12.339022 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b380853-7ffe-4cac-a485-b573aa6fdcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:12.339929 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b380853-7ffe-4cac-a485-b573aa6fdcbb None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:12.340217 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b380853-7ffe-4cac-a485-b573aa6fdcbb None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:12.340498 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b380853-7ffe-4cac-a485-b573aa6fdcbb None admin] PATH_INFO: `/v3/users/e9d06cdd6d164fe8a90b30959493e066` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:12.345352 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:12.347060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b380853-7ffe-4cac-a485-b573aa6fdcbb None admin] RBAC: Authorizing `identity:get_user(user_id=e9d06cdd6d164fe8a90b30959493e066)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:12.348547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b380853-7ffe-4cac-a485-b573aa6fdcbb None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:12.350572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 136/287] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:53:12 2022] GET /identity/v3/users/e9d06cdd6d164fe8a90b30959493e066 => generated 297 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:12.359269 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b380853-7ffe-4cac-a485-b573aa6fdcbb None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:12.394535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709c4bb-d824-4d27-bf24-090974434d2f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:12.434228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a0414-22e3-4d36-a2f6-1088d18973ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:12.434933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a0414-22e3-4d36-a2f6-1088d18973ec None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:12.435528 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a0414-22e3-4d36-a2f6-1088d18973ec None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:12.435824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6a0414-22e3-4d36-a2f6-1088d18973ec None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:12.436594 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a0414-22e3-4d36-a2f6-1088d18973ec None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:12.436909 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a0414-22e3-4d36-a2f6-1088d18973ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:12.438926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b6a0414-22e3-4d36-a2f6-1088d18973ec None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:12.453292 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 152/288] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:53:12 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:12.454602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a0414-22e3-4d36-a2f6-1088d18973ec None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:12.567264 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6a0414-22e3-4d36-a2f6-1088d18973ec None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:12.584037 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:12.606131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b774628-5e1c-47b8-8082-11d6d1ec465d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:12.607188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b774628-5e1c-47b8-8082-11d6d1ec465d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:12.607484 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b774628-5e1c-47b8-8082-11d6d1ec465d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:12.607770 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b774628-5e1c-47b8-8082-11d6d1ec465d None admin] PATH_INFO: `/v3/users/05bf542db18b415087bef3ca1a0395c8` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:12.608984 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b774628-5e1c-47b8-8082-11d6d1ec465d None admin] RBAC: Authorizing `identity:get_user(user_id=05bf542db18b415087bef3ca1a0395c8)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:12.611119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b774628-5e1c-47b8-8082-11d6d1ec465d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:12.617458 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 153/289] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:53:12 2022] GET /identity/v3/users/05bf542db18b415087bef3ca1a0395c8 => generated 297 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:12.618702 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b774628-5e1c-47b8-8082-11d6d1ec465d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:12.736474 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b774628-5e1c-47b8-8082-11d6d1ec465d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:12.772347 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b380853-7ffe-4cac-a485-b573aa6fdcbb None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:12.772922 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec3ba1-c0ac-4fa8-9a4c-637eb756b834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:12.773223 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec3ba1-c0ac-4fa8-9a4c-637eb756b834 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:12.773411 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec3ba1-c0ac-4fa8-9a4c-637eb756b834 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:12.773698 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efec3ba1-c0ac-4fa8-9a4c-637eb756b834 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:12.834765 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:12.850422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec3ba1-c0ac-4fa8-9a4c-637eb756b834 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:12.852389 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efec3ba1-c0ac-4fa8-9a4c-637eb756b834 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:12.908137 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ebbe0-2f08-464a-a6c3-e1dacd9c20df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:12.909037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ebbe0-2f08-464a-a6c3-e1dacd9c20df None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:12.909326 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ebbe0-2f08-464a-a6c3-e1dacd9c20df None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:12.909608 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-461ebbe0-2f08-464a-a6c3-e1dacd9c20df None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:12.910870 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ebbe0-2f08-464a-a6c3-e1dacd9c20df None admin] RBAC: Authorizing `identity:get_project(project_id=34304bd3ae654c1c976f55ca788ae592)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:12.912338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-461ebbe0-2f08-464a-a6c3-e1dacd9c20df None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:12.914422 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 137/290] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:12 2022] GET /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592 => generated 302 bytes in 555 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:12.923565 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ebbe0-2f08-464a-a6c3-e1dacd9c20df None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:12.957157 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:12.980552 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 154/291] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 362 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:12.995733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec3ba1-c0ac-4fa8-9a4c-637eb756b834 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:13.050882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-461ebbe0-2f08-464a-a6c3-e1dacd9c20df None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:13.085368 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:13.088505 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc632e9-b67b-407d-94c0-799285078e87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:13.089492 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc632e9-b67b-407d-94c0-799285078e87 None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:13.089944 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc632e9-b67b-407d-94c0-799285078e87 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:13.090408 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fc632e9-b67b-407d-94c0-799285078e87 None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592/users/e9d06cdd6d164fe8a90b30959493e066/roles/3a110b9d612545b09bf08554a2dd7eac` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:13.092582 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc632e9-b67b-407d-94c0-799285078e87 None admin] RBAC: Authorizing `identity:create_grant(project_id=34304bd3ae654c1c976f55ca788ae592, user_id=e9d06cdd6d164fe8a90b30959493e066, role_id=3a110b9d612545b09bf08554a2dd7eac)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:13.094628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fc632e9-b67b-407d-94c0-799285078e87 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:13.108427 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efec3ba1-c0ac-4fa8-9a4c-637eb756b834 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:13.108989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 138/292] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:53:12 2022] PUT /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592/users/e9d06cdd6d164fe8a90b30959493e066/roles/3a110b9d612545b09bf08554a2dd7eac => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:53:13.160315 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8771a9-a18a-47bf-b107-50b356c64298 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:13.161240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8771a9-a18a-47bf-b107-50b356c64298 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:13.161523 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8771a9-a18a-47bf-b107-50b356c64298 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:13.161800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8771a9-a18a-47bf-b107-50b356c64298 None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:13.163099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8771a9-a18a-47bf-b107-50b356c64298 None admin] RBAC: Authorizing `identity:get_project(project_id=34304bd3ae654c1c976f55ca788ae592)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:13.164567 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8771a9-a18a-47bf-b107-50b356c64298 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:13.166350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 155/293] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:12 2022] GET /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:13.172692 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc632e9-b67b-407d-94c0-799285078e87 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:13.305221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:13.333504 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fc632e9-b67b-407d-94c0-799285078e87 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:13.371834 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dfe721c-475a-4184-83b4-da9fa2bd1722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:13.372417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe721c-475a-4184-83b4-da9fa2bd1722 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:13.372704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe721c-475a-4184-83b4-da9fa2bd1722 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:13.372976 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dfe721c-475a-4184-83b4-da9fa2bd1722 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:13.373831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe721c-475a-4184-83b4-da9fa2bd1722 None admin] RBAC: Adding query filter params (role.id=827f1f1239754bf797e64f270ba1a80c, scope.project.id=34304bd3ae654c1c976f55ca788ae592, user.id=05bf542db18b415087bef3ca1a0395c8) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:13.374143 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe721c-475a-4184-83b4-da9fa2bd1722 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:13.375679 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dfe721c-475a-4184-83b4-da9fa2bd1722 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:13.386431 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 139/294] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:53:13 2022] GET /identity/v3/role_assignments?user.id=05bf542db18b415087bef3ca1a0395c8&scope.project.id=34304bd3ae654c1c976f55ca788ae592&role.id=827f1f1239754bf797e64f270ba1a80c => generated 608 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:13.614544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c4988-2772-40a1-8b53-533bbe1ce212 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:13.615229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c4988-2772-40a1-8b53-533bbe1ce212 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:13.615229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c4988-2772-40a1-8b53-533bbe1ce212 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:13.616623 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 156/295] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:13.622111 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91852feb-bdbb-4be6-a28c-ce42a29b6804 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:13.622379 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91852feb-bdbb-4be6-a28c-ce42a29b6804 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:13.622636 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91852feb-bdbb-4be6-a28c-ce42a29b6804 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:13.626376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5c5b7-b99b-43a3-8f33-c268debc5e0a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:13.626664 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5c5b7-b99b-43a3-8f33-c268debc5e0a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:13.626937 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a5c5b7-b99b-43a3-8f33-c268debc5e0a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:13.628410 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 157/296] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:13.633802 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1882729-d140-436e-afc6-755b34087637 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:13.634087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1882729-d140-436e-afc6-755b34087637 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:13.634426 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1882729-d140-436e-afc6-755b34087637 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:13.643259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-91852feb-bdbb-4be6-a28c-ce42a29b6804 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:13.644466 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-91852feb-bdbb-4be6-a28c-ce42a29b6804 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:13.655118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b1882729-d140-436e-afc6-755b34087637 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:13.656356 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b1882729-d140-436e-afc6-755b34087637 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:13.718600 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 140/297] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:13.723897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f85659-694c-4202-a090-b9c130874ef0 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:13.724263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f85659-694c-4202-a090-b9c130874ef0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:13.724417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f85659-694c-4202-a090-b9c130874ef0 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:13.726205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 141/298] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:13 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:13.735006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df975635-29b6-4582-96d0-0618039cb235 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:13.735311 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df975635-29b6-4582-96d0-0618039cb235 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:13.735574 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df975635-29b6-4582-96d0-0618039cb235 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:13.749030 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 158/299] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:53:13.750954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1882729-d140-436e-afc6-755b34087637 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:13.755533 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-df975635-29b6-4582-96d0-0618039cb235 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:13.756323 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-df975635-29b6-4582-96d0-0618039cb235 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:13.830956 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 142/300] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:13.831823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df975635-29b6-4582-96d0-0618039cb235 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:13.834235 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:13.864317 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:13.888006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1882729-d140-436e-afc6-755b34087637 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:13.928635 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc7517-f5ab-468c-8048-0972f56021fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:13.929547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc7517-f5ab-468c-8048-0972f56021fa None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:13.929861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc7517-f5ab-468c-8048-0972f56021fa None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:13.930498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fc7517-f5ab-468c-8048-0972f56021fa None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:13.931695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc7517-f5ab-468c-8048-0972f56021fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:13.933380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12fc7517-f5ab-468c-8048-0972f56021fa None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:13.935048 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 159/301] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:13.935929 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc7517-f5ab-468c-8048-0972f56021fa None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:13.939808 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df975635-29b6-4582-96d0-0618039cb235 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:13.975544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af978221-56f0-4296-88ac-27a4193f3a64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:13.976326 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af978221-56f0-4296-88ac-27a4193f3a64 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:13.976651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af978221-56f0-4296-88ac-27a4193f3a64 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:13.976893 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af978221-56f0-4296-88ac-27a4193f3a64 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:13.987906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af978221-56f0-4296-88ac-27a4193f3a64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:13.995751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-af978221-56f0-4296-88ac-27a4193f3a64 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 13:53:13.995751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:13.995751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:13.995751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:13.995751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:13.995751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:13.995751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:13.995751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:13.995751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:13.995751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:13.995751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:13.995751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:13.995751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:13.995751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:13.995751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:13.995751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:13.996832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:13.996832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:13.996832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:13.996832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:13.996832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:13.996832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:13.996832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:13.996832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:13.996832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:13.996832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:13.996832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:13.996832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:13.996832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:13.996832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:13.996832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:13.996832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:13.997623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:13.997623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:13.997623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:13.997623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:13.997623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:13.997623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:13.997623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:13.997623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:13.997623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:13.997623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:13.997623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 13:53:13.997623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:13.997623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 143/302] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:13 2022] GET /identity/v3/roles/reader => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:13.998291 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af978221-56f0-4296-88ac-27a4193f3a64 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:14.052925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fc7517-f5ab-468c-8048-0972f56021fa None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:14.066768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:14.095420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef0fda-081b-4a9c-9282-649ed53ed02d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:14.096234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef0fda-081b-4a9c-9282-649ed53ed02d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:14.096523 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef0fda-081b-4a9c-9282-649ed53ed02d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:14.096809 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcef0fda-081b-4a9c-9282-649ed53ed02d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:14.107656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcef0fda-081b-4a9c-9282-649ed53ed02d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:14.110937 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af978221-56f0-4296-88ac-27a4193f3a64 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:14.116204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-dcef0fda-081b-4a9c-9282-649ed53ed02d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:14.116204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:14.116204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:14.116204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:14.116204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:14.116204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:14.116204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:14.116204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:14.116204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:14.116204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:14.116204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:14.116204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:14.116204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:14.116204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:14.116204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:14.116204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:14.117277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:14.117277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:14.117277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:14.117277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:14.117277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:14.117277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:14.117277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:14.117277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:14.117277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:14.117277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:14.117277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:14.117277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:14.117277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:14.117277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:14.117277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:14.117277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:14.118249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:14.118249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:14.118249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:14.118249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:14.118249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:14.118249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:14.118249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:14.118249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:14.118249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:14.118249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:14.118249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:14.118249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:14.122481 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 160/303] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:14.123343 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef0fda-081b-4a9c-9282-649ed53ed02d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:14.135404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:14.151896 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dec488-50b0-4367-9f2e-80d92c10076b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:14.152488 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dec488-50b0-4367-9f2e-80d92c10076b None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:14.152779 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dec488-50b0-4367-9f2e-80d92c10076b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:14.153057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4dec488-50b0-4367-9f2e-80d92c10076b None admin] PATH_INFO: `/v3/groups` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:14.153764 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dec488-50b0-4367-9f2e-80d92c10076b None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:14.155250 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4dec488-50b0-4367-9f2e-80d92c10076b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:14.178345 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 144/304] 10.222.0.49 () {60 vars in 1134 bytes} [Mon Jul 11 13:53:13 2022] POST /identity/v3/groups => generated 229 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:53:14.178345 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dec488-50b0-4367-9f2e-80d92c10076b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:14.250778 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcef0fda-081b-4a9c-9282-649ed53ed02d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:14.291040 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e2729-09c5-4974-b4bb-165b4fb67f5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:14.291939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e2729-09c5-4974-b4bb-165b4fb67f5b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:14.292338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e2729-09c5-4974-b4bb-165b4fb67f5b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:14.292799 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439e2729-09c5-4974-b4bb-165b4fb67f5b None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:14.293699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e2729-09c5-4974-b4bb-165b4fb67f5b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:14.294132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e2729-09c5-4974-b4bb-165b4fb67f5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:14.295708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-439e2729-09c5-4974-b4bb-165b4fb67f5b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:14.307263 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 161/305] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:53:14 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:14.312704 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e2729-09c5-4974-b4bb-165b4fb67f5b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:14.342454 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4dec488-50b0-4367-9f2e-80d92c10076b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:14.370259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:14.388109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92502fb-ee26-4474-9262-19bf5f792a43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:14.389610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92502fb-ee26-4474-9262-19bf5f792a43 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:14.390380 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92502fb-ee26-4474-9262-19bf5f792a43 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:14.390896 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d92502fb-ee26-4474-9262-19bf5f792a43 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:14.391830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92502fb-ee26-4474-9262-19bf5f792a43 None admin] RBAC: Adding query filter params (name=member) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:14.392145 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92502fb-ee26-4474-9262-19bf5f792a43 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:14.393893 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d92502fb-ee26-4474-9262-19bf5f792a43 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:14.405217 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 145/306] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:53:14 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:14.411453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92502fb-ee26-4474-9262-19bf5f792a43 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:14.426700 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:14.461754 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439e2729-09c5-4974-b4bb-165b4fb67f5b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:14.503604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e12c9-d387-499a-8588-1f7dd29131e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:14.504766 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e12c9-d387-499a-8588-1f7dd29131e9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:14.505204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e12c9-d387-499a-8588-1f7dd29131e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:14.505641 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e12c9-d387-499a-8588-1f7dd29131e9 None admin] PATH_INFO: `/v3/users/3c426ec261a54ecbb833a5a8d65c3fd7` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:14.521433 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879e12c9-d387-499a-8588-1f7dd29131e9 None admin] RBAC: Authorizing `identity:get_user(user_id=3c426ec261a54ecbb833a5a8d65c3fd7)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:14.523284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-879e12c9-d387-499a-8588-1f7dd29131e9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:14.525595 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 162/307] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:53:14 2022] GET /identity/v3/users/3c426ec261a54ecbb833a5a8d65c3fd7 => generated 293 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:14.532925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e12c9-d387-499a-8588-1f7dd29131e9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:14.535018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d92502fb-ee26-4474-9262-19bf5f792a43 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:14.582959 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd2cf3-0b02-477b-a034-797cf703e35a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:14.584029 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd2cf3-0b02-477b-a034-797cf703e35a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:14.584467 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd2cf3-0b02-477b-a034-797cf703e35a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:14.584847 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bdd2cf3-0b02-477b-a034-797cf703e35a None admin] PATH_INFO: `/v3/users/b39c7b757ef94efab0737d70ad55f5f5` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:14.586235 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd2cf3-0b02-477b-a034-797cf703e35a None admin] RBAC: Authorizing `identity:get_user(user_id=b39c7b757ef94efab0737d70ad55f5f5)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:14.587816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bdd2cf3-0b02-477b-a034-797cf703e35a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:14.595218 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 146/308] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:53:14 2022] GET /identity/v3/users/b39c7b757ef94efab0737d70ad55f5f5 => generated 293 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:14.596863 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd2cf3-0b02-477b-a034-797cf703e35a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:14.666792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:14.667728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e12c9-d387-499a-8588-1f7dd29131e9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:14.703962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:14.715763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a2e633-cff7-4b89-9a4f-8bdc8011b1c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:14.716474 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2e633-cff7-4b89-9a4f-8bdc8011b1c2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:14.716760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2e633-cff7-4b89-9a4f-8bdc8011b1c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:14.717036 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a2e633-cff7-4b89-9a4f-8bdc8011b1c2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:14.718436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2e633-cff7-4b89-9a4f-8bdc8011b1c2 None admin] RBAC: Adding query filter params (role.id=3a110b9d612545b09bf08554a2dd7eac, user.id=3c426ec261a54ecbb833a5a8d65c3fd7, scope.system=all) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:14.718833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2e633-cff7-4b89-9a4f-8bdc8011b1c2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:14.721132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a2e633-cff7-4b89-9a4f-8bdc8011b1c2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:14.728954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 163/309] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Jul 11 13:53:14 2022] GET /identity/v3/role_assignments?user.id=3c426ec261a54ecbb833a5a8d65c3fd7&scope.system=all&role.id=3a110b9d612545b09bf08554a2dd7eac => generated 226 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:14.738996 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bdd2cf3-0b02-477b-a034-797cf703e35a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:14.778891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da612f47-328a-4370-98ab-0104f48e5744 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:14.780156 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da612f47-328a-4370-98ab-0104f48e5744 None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:14.780672 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da612f47-328a-4370-98ab-0104f48e5744 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:14.781153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da612f47-328a-4370-98ab-0104f48e5744 None admin] PATH_INFO: `/v3/system/users/b39c7b757ef94efab0737d70ad55f5f5/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:14.784002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da612f47-328a-4370-98ab-0104f48e5744 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=b39c7b757ef94efab0737d70ad55f5f5, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:14.785708 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da612f47-328a-4370-98ab-0104f48e5744 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:14.799853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 147/310] 10.222.0.49 () {58 vars in 1334 bytes} [Mon Jul 11 13:53:14 2022] PUT /identity/v3/system/users/b39c7b757ef94efab0737d70ad55f5f5/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:53:15.414317 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c428c2-5b31-4e74-8c6a-b262a8d64844 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:15.414646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c428c2-5b31-4e74-8c6a-b262a8d64844 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:15.415047 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6c428c2-5b31-4e74-8c6a-b262a8d64844 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:15.416435 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 164/311] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:15.421769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f4b46-90ce-4286-af41-0239d9ca9ace None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:15.422768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f4b46-90ce-4286-af41-0239d9ca9ace None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:15.422939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99f4b46-90ce-4286-af41-0239d9ca9ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:15.442330 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b99f4b46-90ce-4286-af41-0239d9ca9ace None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:15.443544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b99f4b46-90ce-4286-af41-0239d9ca9ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:15.514768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 148/312] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:15.539068 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c428c2-5b31-4e74-8c6a-b262a8d64844 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:15.649371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6c428c2-5b31-4e74-8c6a-b262a8d64844 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:15.685950 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474f7a4-3179-4f24-8d99-25a4ecbda7d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:15.687460 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474f7a4-3179-4f24-8d99-25a4ecbda7d4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:15.688039 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474f7a4-3179-4f24-8d99-25a4ecbda7d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:15.688505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9474f7a4-3179-4f24-8d99-25a4ecbda7d4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:15.699426 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9474f7a4-3179-4f24-8d99-25a4ecbda7d4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:15.707423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-9474f7a4-3179-4f24-8d99-25a4ecbda7d4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 13:53:15.707423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:15.707423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:15.707423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:15.707423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:15.707423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:15.707423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:15.707423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:15.707423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:15.707423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:15.707423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:15.707423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:15.707423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:15.707423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:15.707423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:15.707423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:15.708272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:15.708272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:15.708272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:15.708272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:15.708272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:15.708272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:15.708272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:15.708272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:15.708272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:15.708272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:15.708272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:15.708272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:15.708272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:15.708272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:15.708272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:15.708272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:15.709028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:15.709028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:15.709028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:15.709028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:15.709028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:15.709028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:15.709028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:15.709028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:15.709028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:15.709028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:15.709028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 13:53:15.709028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:15.711354 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 165/313] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:15 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:15.717331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f4b46-90ce-4286-af41-0239d9ca9ace None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:15.774872 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:15.825252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99f4b46-90ce-4286-af41-0239d9ca9ace None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:15.862775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8580f2-860c-4de9-8959-e7bc155c593d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:15.863470 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8580f2-860c-4de9-8959-e7bc155c593d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:15.863760 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8580f2-860c-4de9-8959-e7bc155c593d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:15.864044 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b8580f2-860c-4de9-8959-e7bc155c593d None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:15.864828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8580f2-860c-4de9-8959-e7bc155c593d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:15.865143 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8580f2-860c-4de9-8959-e7bc155c593d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:15.866597 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b8580f2-860c-4de9-8959-e7bc155c593d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:15.876487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 149/314] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:53:15 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:15.881944 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474f7a4-3179-4f24-8d99-25a4ecbda7d4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:15.941813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:15.995278 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9474f7a4-3179-4f24-8d99-25a4ecbda7d4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:16.031664 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f66b54-d0a9-4ade-ab23-29823bc7e0db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:16.032915 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f66b54-d0a9-4ade-ab23-29823bc7e0db None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:16.033391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f66b54-d0a9-4ade-ab23-29823bc7e0db None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:16.033854 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f66b54-d0a9-4ade-ab23-29823bc7e0db None admin] PATH_INFO: `/v3/users/e9d06cdd6d164fe8a90b30959493e066` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:16.035272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f66b54-d0a9-4ade-ab23-29823bc7e0db None admin] RBAC: Authorizing `identity:get_user(user_id=e9d06cdd6d164fe8a90b30959493e066)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:16.036889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f66b54-d0a9-4ade-ab23-29823bc7e0db None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:16.038763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 166/315] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:53:15 2022] GET /identity/v3/users/e9d06cdd6d164fe8a90b30959493e066 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:16.047741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8580f2-860c-4de9-8959-e7bc155c593d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:16.160941 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b8580f2-860c-4de9-8959-e7bc155c593d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:16.170407 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:16.198046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e30dd-3585-4769-8ba9-c77cda9fe89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:16.198654 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e30dd-3585-4769-8ba9-c77cda9fe89e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:16.198950 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e30dd-3585-4769-8ba9-c77cda9fe89e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:16.199235 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-521e30dd-3585-4769-8ba9-c77cda9fe89e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:16.269674 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e30dd-3585-4769-8ba9-c77cda9fe89e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:16.271298 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-521e30dd-3585-4769-8ba9-c77cda9fe89e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:16.329047 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e6074-8da6-4788-99be-33c931560cee None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:16.329364 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e6074-8da6-4788-99be-33c931560cee None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:16.329627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219e6074-8da6-4788-99be-33c931560cee None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:16.331348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 167/316] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:53:16.336705 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b29de-ad89-4e7d-9532-3f0e840b00e4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:16.337000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b29de-ad89-4e7d-9532-3f0e840b00e4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:16.337288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0b29de-ad89-4e7d-9532-3f0e840b00e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:16.355821 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9a0b29de-ad89-4e7d-9532-3f0e840b00e4 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:16.357247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9a0b29de-ad89-4e7d-9532-3f0e840b00e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:16.376481 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 150/317] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:16 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:16.382339 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e30dd-3585-4769-8ba9-c77cda9fe89e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:16.392354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:16.458542 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 168/318] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:53:16.483872 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b29de-ad89-4e7d-9532-3f0e840b00e4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:16.510939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-521e30dd-3585-4769-8ba9-c77cda9fe89e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:16.544651 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:16.566569 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25ebcb-89d9-455e-8d91-49056d48900c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:16.568153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25ebcb-89d9-455e-8d91-49056d48900c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:16.568659 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25ebcb-89d9-455e-8d91-49056d48900c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:16.569526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f25ebcb-89d9-455e-8d91-49056d48900c None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:16.572229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25ebcb-89d9-455e-8d91-49056d48900c None admin] RBAC: Authorizing `identity:get_project(project_id=34304bd3ae654c1c976f55ca788ae592)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:16.574100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f25ebcb-89d9-455e-8d91-49056d48900c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:16.582422 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 151/319] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:16 2022] GET /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592 => generated 302 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:16.584393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25ebcb-89d9-455e-8d91-49056d48900c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:16.617758 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0b29de-ad89-4e7d-9532-3f0e840b00e4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:16.665966 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc62188-c958-4e86-b3ee-c982e4c8a68b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:16.667126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc62188-c958-4e86-b3ee-c982e4c8a68b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:16.667609 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc62188-c958-4e86-b3ee-c982e4c8a68b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:16.668483 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdc62188-c958-4e86-b3ee-c982e4c8a68b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:16.669838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc62188-c958-4e86-b3ee-c982e4c8a68b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:16.670170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:16.672321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdc62188-c958-4e86-b3ee-c982e4c8a68b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:16.679425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 169/320] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:16.680761 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc62188-c958-4e86-b3ee-c982e4c8a68b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:16.706538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f25ebcb-89d9-455e-8d91-49056d48900c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:16.753234 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f64a6132-3420-48c9-802c-1b9ea45b66b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:16.754839 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a6132-3420-48c9-802c-1b9ea45b66b8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:16.755454 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a6132-3420-48c9-802c-1b9ea45b66b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:16.756020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f64a6132-3420-48c9-802c-1b9ea45b66b8 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:16.757054 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a6132-3420-48c9-802c-1b9ea45b66b8 None admin] RBAC: Adding query filter params (role.id=3a110b9d612545b09bf08554a2dd7eac, scope.project.id=34304bd3ae654c1c976f55ca788ae592, user.id=e9d06cdd6d164fe8a90b30959493e066) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:16.757553 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a6132-3420-48c9-802c-1b9ea45b66b8 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:16.760105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f64a6132-3420-48c9-802c-1b9ea45b66b8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:16.769026 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 152/321] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:53:16 2022] GET /identity/v3/role_assignments?user.id=e9d06cdd6d164fe8a90b30959493e066&scope.project.id=34304bd3ae654c1c976f55ca788ae592&role.id=3a110b9d612545b09bf08554a2dd7eac => generated 608 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:16.795716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3fa8f-b106-4986-a378-c7be3798bb2f None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:16.796414 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3fa8f-b106-4986-a378-c7be3798bb2f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:16.797634 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a3fa8f-b106-4986-a378-c7be3798bb2f None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:16.800262 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 153/322] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:53:16.806587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95f924-9ec4-458b-97ad-129090c1bec4 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:16.807209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95f924-9ec4-458b-97ad-129090c1bec4 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:16.807802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc95f924-9ec4-458b-97ad-129090c1bec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:16.811918 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdc62188-c958-4e86-b3ee-c982e4c8a68b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:16.827748 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-dc95f924-9ec4-458b-97ad-129090c1bec4 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:16.829694 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dc95f924-9ec4-458b-97ad-129090c1bec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:16.833298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:16.852470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec0a688-9fd3-428c-a7be-7f163c36c83c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:16.853438 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0a688-9fd3-428c-a7be-7f163c36c83c None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:16.853438 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0a688-9fd3-428c-a7be-7f163c36c83c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:16.853785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec0a688-9fd3-428c-a7be-7f163c36c83c None admin] PATH_INFO: `/v3/groups` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:16.854527 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0a688-9fd3-428c-a7be-7f163c36c83c None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:16.856075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec0a688-9fd3-428c-a7be-7f163c36c83c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:16.882597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 170/323] 10.222.0.49 () {60 vars in 1134 bytes} [Mon Jul 11 13:53:16 2022] POST /identity/v3/groups => generated 226 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:53:16.942931 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 154/324] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:53:16.948089 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9864f-03f0-4a71-a01c-eddb144dd0f3 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:16.948522 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9864f-03f0-4a71-a01c-eddb144dd0f3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:16.948953 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74c9864f-03f0-4a71-a01c-eddb144dd0f3 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:16.951240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 171/325] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:16 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:16.958716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0096a7-d541-4319-8159-cd0debaf4ec2 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:16.959235 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0096a7-d541-4319-8159-cd0debaf4ec2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:16.959663 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0096a7-d541-4319-8159-cd0debaf4ec2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:16.963789 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9864f-03f0-4a71-a01c-eddb144dd0f3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:16.983258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-0e0096a7-d541-4319-8159-cd0debaf4ec2 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:16.985158 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0e0096a7-d541-4319-8159-cd0debaf4ec2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:17.070397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:17.078365 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 155/326] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:17.094193 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74c9864f-03f0-4a71-a01c-eddb144dd0f3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:17.102153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0096a7-d541-4319-8159-cd0debaf4ec2 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:17.141351 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe616485-6356-4735-b9a8-e5ebdfac3cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:17.142162 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe616485-6356-4735-b9a8-e5ebdfac3cd5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:17.142477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe616485-6356-4735-b9a8-e5ebdfac3cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:17.143496 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe616485-6356-4735-b9a8-e5ebdfac3cd5 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:17.160440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe616485-6356-4735-b9a8-e5ebdfac3cd5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:17.170619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-fe616485-6356-4735-b9a8-e5ebdfac3cd5 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 13:53:17.170619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:17.170619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:17.170619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:17.170619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:17.170619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:17.170619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:17.170619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:17.170619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:17.170619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:17.170619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:17.170619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:17.170619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:17.170619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:17.170619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:17.170619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:17.171503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:17.171503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:17.171503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:17.171503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:17.171503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:17.171503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:17.171503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:17.171503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:17.171503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:17.171503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:17.171503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:17.171503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:17.171503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:17.171503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:17.171503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:17.171503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:17.172833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:17.172833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:17.172833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:17.172833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:17.172833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:17.172833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:17.172833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:17.172833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:17.172833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:17.172833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:17.172833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 13:53:17.172833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:17.175790 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 172/327] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:17.181466 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe616485-6356-4735-b9a8-e5ebdfac3cd5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:17.221994 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0096a7-d541-4319-8159-cd0debaf4ec2 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:17.229244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:17.275180 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b4e38-f5d4-426f-91f6-2aaade04fecb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:17.276007 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b4e38-f5d4-426f-91f6-2aaade04fecb None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:17.276300 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b4e38-f5d4-426f-91f6-2aaade04fecb None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:17.277070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b4e38-f5d4-426f-91f6-2aaade04fecb None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:17.291136 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4b4e38-f5d4-426f-91f6-2aaade04fecb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:17.300982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-0f4b4e38-f5d4-426f-91f6-2aaade04fecb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:17.300982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:17.300982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:17.300982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:17.300982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:17.300982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:17.300982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:17.300982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:17.300982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:17.300982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:17.300982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:17.300982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:17.300982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:17.300982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:17.300982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:17.300982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:17.302123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:17.302123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:17.302123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:17.302123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:17.302123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:17.302123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:17.302123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:17.302123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:17.302123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:17.302123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:17.302123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:17.302123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:17.302123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:17.302123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:17.302123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:17.302123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:17.303220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:17.303220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:17.303220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:17.303220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:17.303220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:17.303220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:17.303220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:17.303220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:17.303220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:17.303220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:17.303220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:17.303220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:17.303220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 156/328] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:53:17.309584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b4e38-f5d4-426f-91f6-2aaade04fecb None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:17.310132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe616485-6356-4735-b9a8-e5ebdfac3cd5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:17.361978 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62fa89-b340-43dc-81b7-45e868a6ff59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:17.362730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62fa89-b340-43dc-81b7-45e868a6ff59 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:17.363024 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62fa89-b340-43dc-81b7-45e868a6ff59 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:17.363312 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f62fa89-b340-43dc-81b7-45e868a6ff59 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:17.364086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f62fa89-b340-43dc-81b7-45e868a6ff59 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:17.364402 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f62fa89-b340-43dc-81b7-45e868a6ff59 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:17.365817 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f62fa89-b340-43dc-81b7-45e868a6ff59 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:17.376252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 173/329] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:53:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:17.381651 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62fa89-b340-43dc-81b7-45e868a6ff59 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:17.388817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:17.442065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b4e38-f5d4-426f-91f6-2aaade04fecb None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:17.488290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69722d0e-62a3-4f74-ad0e-de201a74ed45 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:17.489271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69722d0e-62a3-4f74-ad0e-de201a74ed45 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:17.489732 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69722d0e-62a3-4f74-ad0e-de201a74ed45 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:17.490179 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69722d0e-62a3-4f74-ad0e-de201a74ed45 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:17.491175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69722d0e-62a3-4f74-ad0e-de201a74ed45 None admin] RBAC: Adding query filter params (name=member) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:17.491666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69722d0e-62a3-4f74-ad0e-de201a74ed45 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:17.493782 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69722d0e-62a3-4f74-ad0e-de201a74ed45 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:17.506533 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 157/330] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:53:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:17.511341 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69722d0e-62a3-4f74-ad0e-de201a74ed45 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:17.527420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:17.532959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f62fa89-b340-43dc-81b7-45e868a6ff59 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:17.582077 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e8ab6-2f49-4134-9574-e2c9633ea99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:17.582077 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e8ab6-2f49-4134-9574-e2c9633ea99a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:17.582077 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e8ab6-2f49-4134-9574-e2c9633ea99a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:17.582452 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e8ab6-2f49-4134-9574-e2c9633ea99a None admin] PATH_INFO: `/v3/users/3c426ec261a54ecbb833a5a8d65c3fd7` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:17.584131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e8ab6-2f49-4134-9574-e2c9633ea99a None admin] RBAC: Authorizing `identity:get_user(user_id=3c426ec261a54ecbb833a5a8d65c3fd7)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:17.585605 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e8ab6-2f49-4134-9574-e2c9633ea99a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:17.591783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 174/331] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:53:17 2022] GET /identity/v3/users/3c426ec261a54ecbb833a5a8d65c3fd7 => generated 293 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:17.595432 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e8ab6-2f49-4134-9574-e2c9633ea99a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:17.653616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69722d0e-62a3-4f74-ad0e-de201a74ed45 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:17.681715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:17.701708 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e5094-f606-406c-973e-90e784ca854d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:17.703000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e5094-f606-406c-973e-90e784ca854d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:17.704007 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e5094-f606-406c-973e-90e784ca854d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:17.704499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f69e5094-f606-406c-973e-90e784ca854d None admin] PATH_INFO: `/v3/users/b39c7b757ef94efab0737d70ad55f5f5` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:17.705988 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e5094-f606-406c-973e-90e784ca854d None admin] RBAC: Authorizing `identity:get_user(user_id=b39c7b757ef94efab0737d70ad55f5f5)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:17.708237 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f69e5094-f606-406c-973e-90e784ca854d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:17.710986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 158/332] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:53:17 2022] GET /identity/v3/users/b39c7b757ef94efab0737d70ad55f5f5 => generated 293 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:17.717145 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e5094-f606-406c-973e-90e784ca854d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:17.750654 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e8ab6-2f49-4134-9574-e2c9633ea99a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:17.794126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ac56b5-7cfc-4856-8c86-4d22c8c0ae18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:17.795391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac56b5-7cfc-4856-8c86-4d22c8c0ae18 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:17.795882 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac56b5-7cfc-4856-8c86-4d22c8c0ae18 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:17.796222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ac56b5-7cfc-4856-8c86-4d22c8c0ae18 None admin] PATH_INFO: `/v3/system/users/3c426ec261a54ecbb833a5a8d65c3fd7/roles/3a110b9d612545b09bf08554a2dd7eac` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:17.798041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac56b5-7cfc-4856-8c86-4d22c8c0ae18 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=3c426ec261a54ecbb833a5a8d65c3fd7, role_id=3a110b9d612545b09bf08554a2dd7eac)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:17.799541 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ac56b5-7cfc-4856-8c86-4d22c8c0ae18 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:17.813626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 175/333] 10.222.0.49 () {58 vars in 1334 bytes} [Mon Jul 11 13:53:17 2022] PUT /identity/v3/system/users/3c426ec261a54ecbb833a5a8d65c3fd7/roles/3a110b9d612545b09bf08554a2dd7eac => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:53:17.862923 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f69e5094-f606-406c-973e-90e784ca854d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:17.904315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db63b2c-2a4e-4d0f-b78f-e019edbe9e3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:17.905311 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db63b2c-2a4e-4d0f-b78f-e019edbe9e3c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:17.905781 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db63b2c-2a4e-4d0f-b78f-e019edbe9e3c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:17.906906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db63b2c-2a4e-4d0f-b78f-e019edbe9e3c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:17.908135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db63b2c-2a4e-4d0f-b78f-e019edbe9e3c None admin] RBAC: Adding query filter params (role.id=827f1f1239754bf797e64f270ba1a80c, user.id=b39c7b757ef94efab0737d70ad55f5f5, scope.system=all) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:17.908775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db63b2c-2a4e-4d0f-b78f-e019edbe9e3c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:17.910852 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db63b2c-2a4e-4d0f-b78f-e019edbe9e3c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:17.926438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 159/334] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Jul 11 13:53:17 2022] GET /identity/v3/role_assignments?user.id=b39c7b757ef94efab0737d70ad55f5f5&scope.system=all&role.id=827f1f1239754bf797e64f270ba1a80c => generated 510 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:19.708774 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe086043-496d-480c-bf52-b294cda67ced None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:19.709280 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe086043-496d-480c-bf52-b294cda67ced None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:19.709385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe086043-496d-480c-bf52-b294cda67ced None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:19.711016 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 176/335] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:19.716736 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b4077-39a0-4af1-a417-c0cafc2e0e98 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:19.717171 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b4077-39a0-4af1-a417-c0cafc2e0e98 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:19.717354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32b4077-39a0-4af1-a417-c0cafc2e0e98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:19.740510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b32b4077-39a0-4af1-a417-c0cafc2e0e98 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:19.741716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b32b4077-39a0-4af1-a417-c0cafc2e0e98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:19.807818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:19.835951 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 160/336] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:19.872297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe086043-496d-480c-bf52-b294cda67ced None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:19.919771 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:20.005002 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe086043-496d-480c-bf52-b294cda67ced None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:20.049640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-398e64fd-e2f7-4fdd-819b-faf741588b0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:20.050498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e64fd-e2f7-4fdd-819b-faf741588b0e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:20.058339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e64fd-e2f7-4fdd-819b-faf741588b0e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:20.058339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-398e64fd-e2f7-4fdd-819b-faf741588b0e None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:20.070125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-398e64fd-e2f7-4fdd-819b-faf741588b0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:20.079785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-398e64fd-e2f7-4fdd-819b-faf741588b0e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:20.079785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:20.079785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:20.079785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:20.079785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:20.079785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:20.079785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:20.079785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:20.079785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:20.079785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:20.079785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:20.079785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:20.079785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:20.079785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:20.079785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:20.079785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:20.080726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:20.080726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:20.080726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:20.080726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:20.080726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:20.080726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:20.080726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:20.080726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:20.080726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:20.080726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:20.080726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:20.080726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:20.080726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:20.080726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:20.080726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:20.080726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:20.081545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:20.081545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:20.081545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:20.081545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:20.081545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:20.081545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:20.081545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:20.081545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:20.081545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:20.081545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:20.081545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:20.081545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:20.081545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 177/337] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:19 2022] GET /identity/v3/roles/member => generated 84 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:20.091461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b4077-39a0-4af1-a417-c0cafc2e0e98 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:20.236382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32b4077-39a0-4af1-a417-c0cafc2e0e98 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:20.275990 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a4feb4-c959-4500-a146-14d80b338e55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:20.276602 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4feb4-c959-4500-a146-14d80b338e55 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:20.276892 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4feb4-c959-4500-a146-14d80b338e55 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:20.277168 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a4feb4-c959-4500-a146-14d80b338e55 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:20.277936 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4feb4-c959-4500-a146-14d80b338e55 None admin] RBAC: Adding query filter params (name=member) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:20.278249 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4feb4-c959-4500-a146-14d80b338e55 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:20.279672 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28a4feb4-c959-4500-a146-14d80b338e55 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:20.294549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a6ff9-0a1a-4120-8891-cde55732438a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:20.294866 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 161/338] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:53:20 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:20.295022 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a6ff9-0a1a-4120-8891-cde55732438a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:20.295222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4a6ff9-0a1a-4120-8891-cde55732438a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:20.297592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 178/339] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:53:20.301972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca7bb6-9093-478a-a625-5fac3fcac238 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:20.302267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca7bb6-9093-478a-a625-5fac3fcac238 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:20.302478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a6ff9-0a1a-4120-8891-cde55732438a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:20.302638 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eca7bb6-9093-478a-a625-5fac3fcac238 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:20.329513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1eca7bb6-9093-478a-a625-5fac3fcac238 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:20.330757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1eca7bb6-9093-478a-a625-5fac3fcac238 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:20.380395 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:20.431134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 162/340] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:20.455353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca7bb6-9093-478a-a625-5fac3fcac238 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:20.480187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:20.492510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4a6ff9-0a1a-4120-8891-cde55732438a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:20.547844 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e08d9732-7f09-437e-9346-eb5885f2e2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:20.548445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d9732-7f09-437e-9346-eb5885f2e2cb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:20.548733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d9732-7f09-437e-9346-eb5885f2e2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:20.549014 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e08d9732-7f09-437e-9346-eb5885f2e2cb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:20.631031 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eca7bb6-9093-478a-a625-5fac3fcac238 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:20.638094 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d9732-7f09-437e-9346-eb5885f2e2cb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:20.639682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e08d9732-7f09-437e-9346-eb5885f2e2cb None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:20.680366 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626fdf0f-46be-4c4d-9baa-6b7d7f275e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:20.681462 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fdf0f-46be-4c4d-9baa-6b7d7f275e1c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:20.681753 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fdf0f-46be-4c4d-9baa-6b7d7f275e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:20.682038 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626fdf0f-46be-4c4d-9baa-6b7d7f275e1c None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:20.697617 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626fdf0f-46be-4c4d-9baa-6b7d7f275e1c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:20.708933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-626fdf0f-46be-4c4d-9baa-6b7d7f275e1c None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:53:20.708933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:20.708933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:20.708933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:20.708933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:20.708933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:20.708933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:20.708933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:20.708933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:20.708933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:20.708933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:20.708933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:20.708933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:20.708933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:20.708933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:20.708933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:20.709829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:20.709829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:20.709829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:20.709829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:20.709829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:20.709829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:20.709829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:20.709829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:20.709829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:20.709829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:20.709829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:20.709829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:20.709829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:20.709829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:20.709829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:20.709829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:20.710666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:20.710666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:20.710666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:20.710666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:20.710666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:20.710666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:20.710666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:20.710666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:20.710666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:20.710666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:20.710666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:53:20.710666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:20.711825 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 163/341] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:53:20 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:20.713012 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702375da-7e09-4fa4-b0cc-f2fa53f15433 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:20.713307 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702375da-7e09-4fa4-b0cc-f2fa53f15433 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:20.714040 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-702375da-7e09-4fa4-b0cc-f2fa53f15433 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:20.716021 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 164/342] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:20.717065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d7883-5588-4f46-a794-2a7da34f8230 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:20.717677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d7883-5588-4f46-a794-2a7da34f8230 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:20.717677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11d7883-5588-4f46-a794-2a7da34f8230 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:20.719322 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 165/343] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:20.720043 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d7883-5588-4f46-a794-2a7da34f8230 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:20.761014 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:20.775769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:20.790684 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 179/344] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 488 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 13:53:20.792008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25293f88-d5a2-4230-a579-1b83c2138f73 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:20.792300 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25293f88-d5a2-4230-a579-1b83c2138f73 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:20.792578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25293f88-d5a2-4230-a579-1b83c2138f73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:20.817157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-25293f88-d5a2-4230-a579-1b83c2138f73 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:20.819479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-25293f88-d5a2-4230-a579-1b83c2138f73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:20.854063 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11d7883-5588-4f46-a794-2a7da34f8230 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:20.890931 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d8efec2-4788-491b-b076-fc5347bc36fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:20.891966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8efec2-4788-491b-b076-fc5347bc36fe None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:20.892401 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8efec2-4788-491b-b076-fc5347bc36fe None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:20.892857 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8efec2-4788-491b-b076-fc5347bc36fe None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:20.893773 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8efec2-4788-491b-b076-fc5347bc36fe None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:20.894250 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8efec2-4788-491b-b076-fc5347bc36fe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:20.896059 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d8efec2-4788-491b-b076-fc5347bc36fe None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:20.905858 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 166/345] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:20 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:20.907870 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 180/346] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:20.908227 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ccdf9-4f96-4a44-8dab-cbf6c6c80ab7 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:20.908711 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ccdf9-4f96-4a44-8dab-cbf6c6c80ab7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:20.908842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25293f88-d5a2-4230-a579-1b83c2138f73 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:20.909258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4ccdf9-4f96-4a44-8dab-cbf6c6c80ab7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:20.928362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-dd4ccdf9-4f96-4a44-8dab-cbf6c6c80ab7 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:20.930187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dd4ccdf9-4f96-4a44-8dab-cbf6c6c80ab7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:21.011183 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 167/347] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:21.012382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ccdf9-4f96-4a44-8dab-cbf6c6c80ab7 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:21.046749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:21.051702 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25293f88-d5a2-4230-a579-1b83c2138f73 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:21.087508 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:21.101039 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3304496-e3fa-484e-9486-71aa3cf1aba9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:21.101919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3304496-e3fa-484e-9486-71aa3cf1aba9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:21.102202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3304496-e3fa-484e-9486-71aa3cf1aba9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:21.102521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3304496-e3fa-484e-9486-71aa3cf1aba9 None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:21.103991 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3304496-e3fa-484e-9486-71aa3cf1aba9 None admin] RBAC: Authorizing `identity:get_project(project_id=9c32ce113f734d9cbd5e90555ab94cdd)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:21.105247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3304496-e3fa-484e-9486-71aa3cf1aba9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:21.107198 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 181/348] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:20 2022] GET /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd => generated 298 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:21.108627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3304496-e3fa-484e-9486-71aa3cf1aba9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:21.141419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4ccdf9-4f96-4a44-8dab-cbf6c6c80ab7 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:21.178949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8ee71-6c4f-4013-9ea7-df229a7da280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:21.180255 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8ee71-6c4f-4013-9ea7-df229a7da280 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:21.180800 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8ee71-6c4f-4013-9ea7-df229a7da280 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:21.181379 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf8ee71-6c4f-4013-9ea7-df229a7da280 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:21.230620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3304496-e3fa-484e-9486-71aa3cf1aba9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:21.275617 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8ee71-6c4f-4013-9ea7-df229a7da280 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:21.276839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7578eb-0f52-4da8-9bb5-8e426d675be7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:21.277643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7578eb-0f52-4da8-9bb5-8e426d675be7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:21.277928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7578eb-0f52-4da8-9bb5-8e426d675be7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:21.278177 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf8ee71-6c4f-4013-9ea7-df229a7da280 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:21.278277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7578eb-0f52-4da8-9bb5-8e426d675be7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:21.289754 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7578eb-0f52-4da8-9bb5-8e426d675be7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:21.298603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-3d7578eb-0f52-4da8-9bb5-8e426d675be7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:21.298603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:21.298603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:21.298603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:21.298603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:21.298603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:21.298603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:21.298603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:21.298603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:21.298603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:21.298603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:21.298603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:21.298603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:21.298603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:21.298603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:21.298603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:21.299547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:21.299547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:21.299547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:21.299547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:21.299547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:21.299547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:21.299547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:21.299547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:21.299547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:21.299547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:21.299547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:21.299547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:21.299547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:21.299547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:21.299547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:21.299547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:21.300476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:21.300476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:21.300476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:21.300476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:21.300476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:21.300476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:21.300476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:21.300476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:21.300476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:21.300476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:21.300476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:21.300476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:21.300476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 182/349] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:21.301144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7578eb-0f52-4da8-9bb5-8e426d675be7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:21.318741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:21.361635 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:21.408093 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 168/350] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 396 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:21.409255 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8ee71-6c4f-4013-9ea7-df229a7da280 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:21.433637 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7578eb-0f52-4da8-9bb5-8e426d675be7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:21.478409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525ddb7-19f5-4cd1-a309-03391329b956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:21.479219 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525ddb7-19f5-4cd1-a309-03391329b956 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:21.479505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525ddb7-19f5-4cd1-a309-03391329b956 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:21.479779 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1525ddb7-19f5-4cd1-a309-03391329b956 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:21.496034 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1525ddb7-19f5-4cd1-a309-03391329b956 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:21.504520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-1525ddb7-19f5-4cd1-a309-03391329b956 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 13:53:21.504520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:21.504520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:21.504520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:21.504520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:21.504520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:21.504520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:21.504520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:21.504520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:21.504520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:21.504520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:21.504520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:21.504520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:21.504520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:21.504520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:21.504520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:21.505422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:21.505422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:21.505422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:21.505422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:21.505422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:21.505422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:21.505422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:21.505422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:21.505422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:21.505422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:21.505422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:21.505422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:21.505422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:21.505422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:21.505422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:21.505422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:21.506200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:21.506200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:21.506200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:21.506200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:21.506200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:21.506200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:21.506200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:21.506200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:21.506200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:21.506200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:21.506200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 11 13:53:21.506200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:21.506200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 183/351] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:21.511001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525ddb7-19f5-4cd1-a309-03391329b956 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:21.555473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf8ee71-6c4f-4013-9ea7-df229a7da280 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:21.584892 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:21.598659 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e2579f-5f35-48aa-b8bc-6fd60934237b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:21.599541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2579f-5f35-48aa-b8bc-6fd60934237b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:21.599822 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2579f-5f35-48aa-b8bc-6fd60934237b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:21.600095 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e2579f-5f35-48aa-b8bc-6fd60934237b None admin] PATH_INFO: `/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:21.611213 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2579f-5f35-48aa-b8bc-6fd60934237b None admin] RBAC: Authorizing `identity:get_group(group_id=1f6dd111ebd740e1a7ae97ca6211ffd0)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:21.613058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e2579f-5f35-48aa-b8bc-6fd60934237b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:21.615060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 169/352] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 11 13:53:21 2022] GET /identity/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:21.617837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cc817-675e-4c19-8139-62349b0a357e None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:21.618394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cc817-675e-4c19-8139-62349b0a357e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:21.618886 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-328cc817-675e-4c19-8139-62349b0a357e None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:21.624384 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 170/353] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:21.624384 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cc817-675e-4c19-8139-62349b0a357e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:21.652240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1525ddb7-19f5-4cd1-a309-03391329b956 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:21.666378 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:21.695326 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450046a8-3b1e-48f2-8e12-e2d982c8d98f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:21.696006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450046a8-3b1e-48f2-8e12-e2d982c8d98f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:21.696295 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450046a8-3b1e-48f2-8e12-e2d982c8d98f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:21.696575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-450046a8-3b1e-48f2-8e12-e2d982c8d98f None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:21.697442 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450046a8-3b1e-48f2-8e12-e2d982c8d98f None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:21.697641 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450046a8-3b1e-48f2-8e12-e2d982c8d98f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:21.699117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-450046a8-3b1e-48f2-8e12-e2d982c8d98f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:21.708081 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 184/354] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:53:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:21.710376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450046a8-3b1e-48f2-8e12-e2d982c8d98f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:21.753093 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-328cc817-675e-4c19-8139-62349b0a357e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:21.795846 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874eedc3-f376-4fb2-bd39-7347973d74c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:21.797338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874eedc3-f376-4fb2-bd39-7347973d74c8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:21.797661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874eedc3-f376-4fb2-bd39-7347973d74c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:21.797946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874eedc3-f376-4fb2-bd39-7347973d74c8 None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:21.799403 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874eedc3-f376-4fb2-bd39-7347973d74c8 None admin] RBAC: Authorizing `identity:get_project(project_id=9c32ce113f734d9cbd5e90555ab94cdd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:21.800904 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874eedc3-f376-4fb2-bd39-7347973d74c8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:21.802752 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 171/355] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:21 2022] GET /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:21.803537 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874eedc3-f376-4fb2-bd39-7347973d74c8 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:21.833026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-450046a8-3b1e-48f2-8e12-e2d982c8d98f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:21.871713 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:21.880486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72c1990a-4e13-470a-b23c-fa88f913a11a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:21.881158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1990a-4e13-470a-b23c-fa88f913a11a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:21.881446 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1990a-4e13-470a-b23c-fa88f913a11a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:21.881730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72c1990a-4e13-470a-b23c-fa88f913a11a None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:21.882524 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1990a-4e13-470a-b23c-fa88f913a11a None admin] RBAC: Adding query filter params (name=reader) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:21.882839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1990a-4e13-470a-b23c-fa88f913a11a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:21.884213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72c1990a-4e13-470a-b23c-fa88f913a11a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:21.894804 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 185/356] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:53:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:21.895967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a442f1e-311e-4939-9dcf-77a99526a643 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:21.896258 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a442f1e-311e-4939-9dcf-77a99526a643 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:21.896541 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a442f1e-311e-4939-9dcf-77a99526a643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:21.913870 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2a442f1e-311e-4939-9dcf-77a99526a643 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:21.915151 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2a442f1e-311e-4939-9dcf-77a99526a643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:21.924079 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874eedc3-f376-4fb2-bd39-7347973d74c8 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:21.949940 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:21.963648 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ff34ae-58ce-4e2d-9f1c-fda1a150e683 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:21.964361 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff34ae-58ce-4e2d-9f1c-fda1a150e683 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:21.965383 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff34ae-58ce-4e2d-9f1c-fda1a150e683 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:21.965383 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ff34ae-58ce-4e2d-9f1c-fda1a150e683 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:21.965734 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff34ae-58ce-4e2d-9f1c-fda1a150e683 None admin] RBAC: Adding query filter params (group.id=1f6dd111ebd740e1a7ae97ca6211ffd0, role.id=827f1f1239754bf797e64f270ba1a80c, scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:21.966969 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff34ae-58ce-4e2d-9f1c-fda1a150e683 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:21.967580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ff34ae-58ce-4e2d-9f1c-fda1a150e683 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:21.974438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 172/357] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 11 13:53:21 2022] GET /identity/v3/role_assignments?group.id=1f6dd111ebd740e1a7ae97ca6211ffd0&scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd&role.id=827f1f1239754bf797e64f270ba1a80c => generated 260 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:21.975570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5d759-4c97-4ece-b25a-01ad012175b6 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:21.975896 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5d759-4c97-4ece-b25a-01ad012175b6 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:21.976549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c5d759-4c97-4ece-b25a-01ad012175b6 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:21.977958 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 173/358] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:21.982881 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5d759-4c97-4ece-b25a-01ad012175b6 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:22.009756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 186/359] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:22.012835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a442f1e-311e-4939-9dcf-77a99526a643 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:22.146205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c5d759-4c97-4ece-b25a-01ad012175b6 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:22.153628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a442f1e-311e-4939-9dcf-77a99526a643 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:22.166295 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:22.191711 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa6357-aa0d-47be-8e16-0d7796e29373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:22.192371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa6357-aa0d-47be-8e16-0d7796e29373 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:22.192651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa6357-aa0d-47be-8e16-0d7796e29373 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:22.192927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8aa6357-aa0d-47be-8e16-0d7796e29373 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:22.195124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523703fa-9378-4361-ac1b-0ae75d9f2509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:22.195983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523703fa-9378-4361-ac1b-0ae75d9f2509 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:22.196269 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523703fa-9378-4361-ac1b-0ae75d9f2509 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:22.196542 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523703fa-9378-4361-ac1b-0ae75d9f2509 None admin] PATH_INFO: `/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:22.199128 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523703fa-9378-4361-ac1b-0ae75d9f2509 None admin] RBAC: Authorizing `identity:get_group(group_id=1f6dd111ebd740e1a7ae97ca6211ffd0)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:22.200595 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523703fa-9378-4361-ac1b-0ae75d9f2509 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:22.207985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 187/360] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 11 13:53:22 2022] GET /identity/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0 => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:22.207985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523703fa-9378-4361-ac1b-0ae75d9f2509 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:22.251743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:22.297340 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa6357-aa0d-47be-8e16-0d7796e29373 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:22.299200 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8aa6357-aa0d-47be-8e16-0d7796e29373 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:22.342143 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523703fa-9378-4361-ac1b-0ae75d9f2509 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:22.398727 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfd400f4-554c-4e42-9938-d00074313535 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:22.399545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd400f4-554c-4e42-9938-d00074313535 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:22.399832 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd400f4-554c-4e42-9938-d00074313535 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:22.400108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfd400f4-554c-4e42-9938-d00074313535 None admin] PATH_INFO: `/v3/users/3c426ec261a54ecbb833a5a8d65c3fd7` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:22.402294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd400f4-554c-4e42-9938-d00074313535 None admin] RBAC: Authorizing `identity:get_user(user_id=3c426ec261a54ecbb833a5a8d65c3fd7)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:22.403025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfd400f4-554c-4e42-9938-d00074313535 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:22.404705 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 188/361] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:53:22 2022] GET /identity/v3/users/3c426ec261a54ecbb833a5a8d65c3fd7 => generated 293 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:22.405903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db0133-b7f1-4096-ae40-b8bedefb8fb6 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:22.407039 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db0133-b7f1-4096-ae40-b8bedefb8fb6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:22.407146 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0db0133-b7f1-4096-ae40-b8bedefb8fb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:22.424822 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b0db0133-b7f1-4096-ae40-b8bedefb8fb6 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:22.426021 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b0db0133-b7f1-4096-ae40-b8bedefb8fb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:22.441482 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 174/362] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 463 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:22.442536 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa6357-aa0d-47be-8e16-0d7796e29373 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:22.470103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:22.505181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 189/363] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:22.506109 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db0133-b7f1-4096-ae40-b8bedefb8fb6 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:22.550051 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:22.575089 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8aa6357-aa0d-47be-8e16-0d7796e29373 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:22.620704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d2bde-cf26-46d5-9d87-d6267a8a17c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:22.621521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d2bde-cf26-46d5-9d87-d6267a8a17c1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:22.621803 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d2bde-cf26-46d5-9d87-d6267a8a17c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:22.622077 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e9d2bde-cf26-46d5-9d87-d6267a8a17c1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:22.634622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e9d2bde-cf26-46d5-9d87-d6267a8a17c1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:22.643596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0db0133-b7f1-4096-ae40-b8bedefb8fb6 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:22.643980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-1e9d2bde-cf26-46d5-9d87-d6267a8a17c1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:53:22.643980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:22.643980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:22.643980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:22.643980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:22.643980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:22.643980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:22.643980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:22.643980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:22.643980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:22.643980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:22.643980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:22.643980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:22.643980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:22.643980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:22.643980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:22.644869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:22.644869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:22.644869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:22.644869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:22.644869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:22.644869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:22.644869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:22.644869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:22.644869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:22.644869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:22.644869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:22.644869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:22.644869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:22.644869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:22.644869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:22.644869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:22.645680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:22.645680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:22.645680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:22.645680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:22.645680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:22.645680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:22.645680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:22.645680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:22.645680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:22.645680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:22.645680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:53:22.645680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:22.645680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 175/364] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:53:22 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:22.646376 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d2bde-cf26-46d5-9d87-d6267a8a17c1 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:22.696645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74203d0-8a8e-46f4-a592-4e8f1a67cc7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:22.697540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74203d0-8a8e-46f4-a592-4e8f1a67cc7a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:22.697975 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74203d0-8a8e-46f4-a592-4e8f1a67cc7a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:22.698434 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74203d0-8a8e-46f4-a592-4e8f1a67cc7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:22.699813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74203d0-8a8e-46f4-a592-4e8f1a67cc7a None admin] RBAC: Adding query filter params (group.id=1f6dd111ebd740e1a7ae97ca6211ffd0, role.id=37f000302df54045892d35f57104660a, scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:22.700293 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74203d0-8a8e-46f4-a592-4e8f1a67cc7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:22.702625 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74203d0-8a8e-46f4-a592-4e8f1a67cc7a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:22.710758 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 190/365] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 11 13:53:22 2022] GET /identity/v3/role_assignments?group.id=1f6dd111ebd740e1a7ae97ca6211ffd0&scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd&role.id=37f000302df54045892d35f57104660a => generated 260 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:22.711612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74203d0-8a8e-46f4-a592-4e8f1a67cc7a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:22.766705 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:22.794272 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e9d2bde-cf26-46d5-9d87-d6267a8a17c1 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:22.855358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05995442-d8a4-4cb0-bd57-51f6a59f2b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:22.856039 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05995442-d8a4-4cb0-bd57-51f6a59f2b64 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:22.856415 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05995442-d8a4-4cb0-bd57-51f6a59f2b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:22.856783 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05995442-d8a4-4cb0-bd57-51f6a59f2b64 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:22.857685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05995442-d8a4-4cb0-bd57-51f6a59f2b64 None admin] RBAC: Adding query filter params (role.id=3a110b9d612545b09bf08554a2dd7eac, user.id=3c426ec261a54ecbb833a5a8d65c3fd7, scope.system=all) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:22.858009 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05995442-d8a4-4cb0-bd57-51f6a59f2b64 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:22.860288 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05995442-d8a4-4cb0-bd57-51f6a59f2b64 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:22.863432 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:22.873638 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74203d0-8a8e-46f4-a592-4e8f1a67cc7a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:22.878659 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 176/366] 10.222.0.49 () {56 vars in 1308 bytes} [Mon Jul 11 13:53:22 2022] GET /identity/v3/role_assignments?user.id=3c426ec261a54ecbb833a5a8d65c3fd7&scope.system=all&role.id=3a110b9d612545b09bf08554a2dd7eac => generated 510 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:22.879566 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05995442-d8a4-4cb0-bd57-51f6a59f2b64 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:22.924097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ae3a9-246c-400f-9d52-d88c7ff3b2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:22.925068 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ae3a9-246c-400f-9d52-d88c7ff3b2ec None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:22.925416 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ae3a9-246c-400f-9d52-d88c7ff3b2ec None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:22.925701 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654ae3a9-246c-400f-9d52-d88c7ff3b2ec None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:22.928106 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ae3a9-246c-400f-9d52-d88c7ff3b2ec None admin] RBAC: Authorizing `identity:get_project(project_id=34304bd3ae654c1c976f55ca788ae592)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:22.929574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-654ae3a9-246c-400f-9d52-d88c7ff3b2ec None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:22.931938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 191/367] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:22 2022] GET /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592 => generated 302 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:22.932882 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ae3a9-246c-400f-9d52-d88c7ff3b2ec None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:22.999004 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05995442-d8a4-4cb0-bd57-51f6a59f2b64 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:23.037355 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42454670-8a64-4119-8eed-fa1c4980587f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:23.038241 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42454670-8a64-4119-8eed-fa1c4980587f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:23.038614 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42454670-8a64-4119-8eed-fa1c4980587f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:23.038903 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42454670-8a64-4119-8eed-fa1c4980587f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:23.042161 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:23.055278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42454670-8a64-4119-8eed-fa1c4980587f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:23.060649 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-654ae3a9-246c-400f-9d52-d88c7ff3b2ec None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:23.063899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-42454670-8a64-4119-8eed-fa1c4980587f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:53:23.063899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:23.063899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:23.063899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:23.063899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:23.063899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:23.063899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:23.063899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:23.063899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:23.063899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:23.063899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:23.063899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:23.063899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:23.063899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:23.063899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:23.063899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:23.065968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:23.065968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:23.065968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:23.065968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:23.065968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:23.065968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:23.065968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:23.065968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:23.065968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:23.065968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:23.065968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:23.065968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:23.065968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:23.065968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:23.065968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:23.065968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:23.067975 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:23.067975 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:23.067975 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:23.067975 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:23.067975 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:23.067975 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:23.067975 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:23.067975 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:23.067975 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:23.067975 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:23.067975 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:53:23.067975 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:23.067975 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 177/368] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:53:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:23.067975 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42454670-8a64-4119-8eed-fa1c4980587f None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:23.130082 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d162df-1079-4bab-ab8c-4f6593926920 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:23.130804 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d162df-1079-4bab-ab8c-4f6593926920 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:23.131150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d162df-1079-4bab-ab8c-4f6593926920 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:23.131430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d162df-1079-4bab-ab8c-4f6593926920 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:23.132239 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d162df-1079-4bab-ab8c-4f6593926920 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:23.132554 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d162df-1079-4bab-ab8c-4f6593926920 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:23.133942 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d162df-1079-4bab-ab8c-4f6593926920 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:23.143898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 192/369] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:22 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:23.148975 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d162df-1079-4bab-ab8c-4f6593926920 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:23.181176 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42454670-8a64-4119-8eed-fa1c4980587f None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:23.192776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:23.228355 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b80763-d59e-417f-9fc1-95fbaa1a11c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:23.229304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b80763-d59e-417f-9fc1-95fbaa1a11c5 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:23.229591 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b80763-d59e-417f-9fc1-95fbaa1a11c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:23.229873 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44b80763-d59e-417f-9fc1-95fbaa1a11c5 None admin] PATH_INFO: `/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:23.231266 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b80763-d59e-417f-9fc1-95fbaa1a11c5 None admin] RBAC: Authorizing `identity:get_group(group_id=1f6dd111ebd740e1a7ae97ca6211ffd0)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:23.232769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44b80763-d59e-417f-9fc1-95fbaa1a11c5 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:23.235105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 178/370] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 11 13:53:23 2022] GET /identity/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0 => generated 226 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:23.235893 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b80763-d59e-417f-9fc1-95fbaa1a11c5 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:23.314613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d162df-1079-4bab-ab8c-4f6593926920 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:23.324834 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:23.363715 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86d7e1-4beb-45e1-938a-139f2d34254c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:23.364404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86d7e1-4beb-45e1-938a-139f2d34254c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:23.364693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86d7e1-4beb-45e1-938a-139f2d34254c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:23.365038 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e86d7e1-4beb-45e1-938a-139f2d34254c None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:23.365918 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86d7e1-4beb-45e1-938a-139f2d34254c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:23.366273 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86d7e1-4beb-45e1-938a-139f2d34254c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:23.367833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e86d7e1-4beb-45e1-938a-139f2d34254c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:23.374055 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44b80763-d59e-417f-9fc1-95fbaa1a11c5 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:23.378328 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 193/371] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:23.379196 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86d7e1-4beb-45e1-938a-139f2d34254c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:23.420193 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f81ee-f2f6-490f-ba89-c7335e27bf8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:23.420905 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f81ee-f2f6-490f-ba89-c7335e27bf8b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:23.421197 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f81ee-f2f6-490f-ba89-c7335e27bf8b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:23.421477 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2f81ee-f2f6-490f-ba89-c7335e27bf8b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:23.492982 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:23.510491 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e86d7e1-4beb-45e1-938a-139f2d34254c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:23.513931 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f81ee-f2f6-490f-ba89-c7335e27bf8b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:23.515475 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2f81ee-f2f6-490f-ba89-c7335e27bf8b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:23.550291 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5d76d-d563-4b0e-ab3f-a731d8a26bf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:23.551692 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5d76d-d563-4b0e-ab3f-a731d8a26bf5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:23.552186 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5d76d-d563-4b0e-ab3f-a731d8a26bf5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:23.552773 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb5d76d-d563-4b0e-ab3f-a731d8a26bf5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:23.555738 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5d76d-d563-4b0e-ab3f-a731d8a26bf5 None admin] RBAC: Adding query filter params (group.id=1f6dd111ebd740e1a7ae97ca6211ffd0, role.id=827f1f1239754bf797e64f270ba1a80c, scope.project.id=34304bd3ae654c1c976f55ca788ae592) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:23.556074 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5d76d-d563-4b0e-ab3f-a731d8a26bf5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:23.557724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb5d76d-d563-4b0e-ab3f-a731d8a26bf5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:23.564081 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 194/372] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 11 13:53:23 2022] GET /identity/v3/role_assignments?group.id=1f6dd111ebd740e1a7ae97ca6211ffd0&scope.project.id=34304bd3ae654c1c976f55ca788ae592&role.id=827f1f1239754bf797e64f270ba1a80c => generated 260 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:23.564999 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5d76d-d563-4b0e-ab3f-a731d8a26bf5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:23.627898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:23.727223 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 179/373] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 491 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:23.733175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f81ee-f2f6-490f-ba89-c7335e27bf8b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:23.774330 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb5d76d-d563-4b0e-ab3f-a731d8a26bf5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:23.823150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939014a2-fd62-4ffc-bb7e-3e8539a5eb21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:23.823749 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939014a2-fd62-4ffc-bb7e-3e8539a5eb21 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:23.824036 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939014a2-fd62-4ffc-bb7e-3e8539a5eb21 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:23.824320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-939014a2-fd62-4ffc-bb7e-3e8539a5eb21 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:23.853096 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:23.876906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2f81ee-f2f6-490f-ba89-c7335e27bf8b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:23.918812 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28322f69-bdbc-4dce-8197-05fbb80d60fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:23.919704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28322f69-bdbc-4dce-8197-05fbb80d60fb None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:23.920022 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28322f69-bdbc-4dce-8197-05fbb80d60fb None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:23.920305 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28322f69-bdbc-4dce-8197-05fbb80d60fb None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:23.921715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28322f69-bdbc-4dce-8197-05fbb80d60fb None admin] RBAC: Authorizing `identity:get_project(project_id=34304bd3ae654c1c976f55ca788ae592)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:23.923355 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28322f69-bdbc-4dce-8197-05fbb80d60fb None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:23.925526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 180/374] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:23 2022] GET /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592 => generated 302 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:23.928709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939014a2-fd62-4ffc-bb7e-3e8539a5eb21 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:23.930455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-939014a2-fd62-4ffc-bb7e-3e8539a5eb21 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:23.931502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28322f69-bdbc-4dce-8197-05fbb80d60fb None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:24.003706 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:24.091973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 195/375] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 527 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:24.097748 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939014a2-fd62-4ffc-bb7e-3e8539a5eb21 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:24.129207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28322f69-bdbc-4dce-8197-05fbb80d60fb None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:24.195923 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:24.204110 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336a862-125a-44c2-bc92-841d589e50ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:24.204979 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336a862-125a-44c2-bc92-841d589e50ae None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:24.205256 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336a862-125a-44c2-bc92-841d589e50ae None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:24.205529 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d336a862-125a-44c2-bc92-841d589e50ae None admin] PATH_INFO: `/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:24.206767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336a862-125a-44c2-bc92-841d589e50ae None admin] RBAC: Authorizing `identity:get_group(group_id=1f6dd111ebd740e1a7ae97ca6211ffd0)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:24.208225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d336a862-125a-44c2-bc92-841d589e50ae None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:24.210248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 181/376] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 11 13:53:23 2022] GET /identity/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0 => generated 226 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:24.218247 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336a862-125a-44c2-bc92-841d589e50ae None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:24.299930 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-939014a2-fd62-4ffc-bb7e-3e8539a5eb21 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:24.356941 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75fc9861-eefc-46f8-a05f-2ba9fb9245c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:24.357747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc9861-eefc-46f8-a05f-2ba9fb9245c0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:24.358033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc9861-eefc-46f8-a05f-2ba9fb9245c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:24.358340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75fc9861-eefc-46f8-a05f-2ba9fb9245c0 None admin] PATH_INFO: `/v3/projects/5898aef39ca74b7d8b99cb1aa700e9c2` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:24.363116 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc9861-eefc-46f8-a05f-2ba9fb9245c0 None admin] RBAC: Authorizing `identity:get_project(project_id=5898aef39ca74b7d8b99cb1aa700e9c2)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:24.365057 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75fc9861-eefc-46f8-a05f-2ba9fb9245c0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:24.366958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 196/377] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:24 2022] GET /identity/v3/projects/5898aef39ca74b7d8b99cb1aa700e9c2 => generated 344 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:24.384964 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7598d4-4a70-4be5-a613-9998c51667b7 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:24.385288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7598d4-4a70-4be5-a613-9998c51667b7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:24.385520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7598d4-4a70-4be5-a613-9998c51667b7 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:24.387858 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 197/378] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:53:24.388708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7598d4-4a70-4be5-a613-9998c51667b7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:24.402665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:24.406520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d336a862-125a-44c2-bc92-841d589e50ae None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:24.466007 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32b24abd-9f04-4039-a14c-a2c738c08d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:24.467214 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b24abd-9f04-4039-a14c-a2c738c08d06 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:24.467726 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b24abd-9f04-4039-a14c-a2c738c08d06 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:24.468206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32b24abd-9f04-4039-a14c-a2c738c08d06 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:24.469754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b24abd-9f04-4039-a14c-a2c738c08d06 None admin] RBAC: Adding query filter params (group.id=1f6dd111ebd740e1a7ae97ca6211ffd0, role.id=37f000302df54045892d35f57104660a, scope.project.id=34304bd3ae654c1c976f55ca788ae592) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:24.470584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b24abd-9f04-4039-a14c-a2c738c08d06 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:24.472729 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32b24abd-9f04-4039-a14c-a2c738c08d06 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:24.481575 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 182/379] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 11 13:53:24 2022] GET /identity/v3/role_assignments?group.id=1f6dd111ebd740e1a7ae97ca6211ffd0&scope.project.id=34304bd3ae654c1c976f55ca788ae592&role.id=37f000302df54045892d35f57104660a => generated 260 bytes in 264 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:24.483991 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b085453-6757-448f-a2ab-d0c8cec25e88 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:24.485040 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b085453-6757-448f-a2ab-d0c8cec25e88 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:24.485523 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b085453-6757-448f-a2ab-d0c8cec25e88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:24.514561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-7b085453-6757-448f-a2ab-d0c8cec25e88 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:24.516739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7b085453-6757-448f-a2ab-d0c8cec25e88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:24.536819 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7598d4-4a70-4be5-a613-9998c51667b7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:24.574405 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:24.611010 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52134d-1a1b-4d4b-906a-5929c210d8b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:24.611843 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52134d-1a1b-4d4b-906a-5929c210d8b8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:24.612125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52134d-1a1b-4d4b-906a-5929c210d8b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:24.612401 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c52134d-1a1b-4d4b-906a-5929c210d8b8 None admin] PATH_INFO: `/v3/groups/701f37c2b4504b3abe423ce570d8f48e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:24.628294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52134d-1a1b-4d4b-906a-5929c210d8b8 None admin] RBAC: Authorizing `identity:get_group(group_id=701f37c2b4504b3abe423ce570d8f48e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:24.629819 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c52134d-1a1b-4d4b-906a-5929c210d8b8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:24.634398 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 198/380] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 11 13:53:24 2022] GET /identity/v3/groups/701f37c2b4504b3abe423ce570d8f48e => generated 229 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:24.639436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52134d-1a1b-4d4b-906a-5929c210d8b8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:24.658831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 183/381] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:24.680744 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b085453-6757-448f-a2ab-d0c8cec25e88 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:24.769669 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c52134d-1a1b-4d4b-906a-5929c210d8b8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:24.792372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:24.800097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b085453-6757-448f-a2ab-d0c8cec25e88 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:24.817400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e72b2-5fe3-4dee-9a17-d60837f7d78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:24.818438 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e72b2-5fe3-4dee-9a17-d60837f7d78f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:24.818983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e72b2-5fe3-4dee-9a17-d60837f7d78f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:24.819406 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101e72b2-5fe3-4dee-9a17-d60837f7d78f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:24.820528 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e72b2-5fe3-4dee-9a17-d60837f7d78f None admin] RBAC: Adding query filter params (group.id=701f37c2b4504b3abe423ce570d8f48e, role.id=6964639a7f6c41e8a6701ef533151c41, scope.project.id=5898aef39ca74b7d8b99cb1aa700e9c2) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:24.821019 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e72b2-5fe3-4dee-9a17-d60837f7d78f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:24.822938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101e72b2-5fe3-4dee-9a17-d60837f7d78f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:24.830797 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 199/382] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 11 13:53:24 2022] GET /identity/v3/role_assignments?group.id=701f37c2b4504b3abe423ce570d8f48e&scope.project.id=5898aef39ca74b7d8b99cb1aa700e9c2&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 260 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:24.848542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9b632-60b1-40ac-ad2b-a6c28c07da7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:24.849838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9b632-60b1-40ac-ad2b-a6c28c07da7d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:24.850353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9b632-60b1-40ac-ad2b-a6c28c07da7d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:24.850802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4e9b632-60b1-40ac-ad2b-a6c28c07da7d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:24.865675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4e9b632-60b1-40ac-ad2b-a6c28c07da7d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:24.874109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-d4e9b632-60b1-40ac-ad2b-a6c28c07da7d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:24.874109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:24.874109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:24.874109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:24.874109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:24.874109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:24.874109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:24.874109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:24.874109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:24.874109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:24.874109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:24.874109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:24.874109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:24.874109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:24.874109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:24.874109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:24.875372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:24.875372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:24.875372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:24.875372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:24.875372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:24.875372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:24.875372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:24.875372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:24.875372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:24.875372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:24.875372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:24.875372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:24.875372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:24.875372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:24.875372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:24.875372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:24.876263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:24.876263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:24.876263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:24.876263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:24.876263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:24.876263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:24.876263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:24.876263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:24.876263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:24.876263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:24.876263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:24.876263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:24.878653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 184/383] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:24.885796 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e72b2-5fe3-4dee-9a17-d60837f7d78f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:24.953298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:25.028527 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101e72b2-5fe3-4dee-9a17-d60837f7d78f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:25.072485 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43286b-451a-4463-87c9-8b050e63a5c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:25.073445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43286b-451a-4463-87c9-8b050e63a5c1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:25.073878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43286b-451a-4463-87c9-8b050e63a5c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:25.074321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c43286b-451a-4463-87c9-8b050e63a5c1 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:25.075246 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43286b-451a-4463-87c9-8b050e63a5c1 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:25.078333 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43286b-451a-4463-87c9-8b050e63a5c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:25.078333 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c43286b-451a-4463-87c9-8b050e63a5c1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:25.086957 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 200/384] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:53:24 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:25.092441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9b632-60b1-40ac-ad2b-a6c28c07da7d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:25.206044 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4e9b632-60b1-40ac-ad2b-a6c28c07da7d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:25.243070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1744b9-2edc-4df4-ae9f-42913e6ee226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:25.243935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1744b9-2edc-4df4-ae9f-42913e6ee226 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:25.244220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1744b9-2edc-4df4-ae9f-42913e6ee226 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:25.244500 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1744b9-2edc-4df4-ae9f-42913e6ee226 None admin] PATH_INFO: `/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:25.245708 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1744b9-2edc-4df4-ae9f-42913e6ee226 None admin] RBAC: Authorizing `identity:get_group(group_id=1f6dd111ebd740e1a7ae97ca6211ffd0)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:25.247196 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1744b9-2edc-4df4-ae9f-42913e6ee226 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:25.249187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 185/385] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 11 13:53:25 2022] GET /identity/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0 => generated 226 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:25.254579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43286b-451a-4463-87c9-8b050e63a5c1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:25.290333 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591823e-d67d-472b-a465-7574806c1e86 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:25.290636 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591823e-d67d-472b-a465-7574806c1e86 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:25.290897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3591823e-d67d-472b-a465-7574806c1e86 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:25.292281 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 186/386] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:25.297698 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3b440-d77e-4f42-b2a6-94a76e41a10d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:25.297989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3b440-d77e-4f42-b2a6-94a76e41a10d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:25.298276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f3b440-d77e-4f42-b2a6-94a76e41a10d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:25.322461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-71f3b440-d77e-4f42-b2a6-94a76e41a10d None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:25.324077 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-71f3b440-d77e-4f42-b2a6-94a76e41a10d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:25.362255 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:25.390166 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c43286b-451a-4463-87c9-8b050e63a5c1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:25.402651 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:25.439307 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528de15-2bf8-4667-a668-d5e34e9947d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:25.440245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528de15-2bf8-4667-a668-d5e34e9947d5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:25.440583 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528de15-2bf8-4667-a668-d5e34e9947d5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:25.440871 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3528de15-2bf8-4667-a668-d5e34e9947d5 None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:25.443026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3528de15-2bf8-4667-a668-d5e34e9947d5 None admin] RBAC: Authorizing `identity:get_project(project_id=9c32ce113f734d9cbd5e90555ab94cdd)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:25.444467 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3528de15-2bf8-4667-a668-d5e34e9947d5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:25.450492 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 201/387] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:25 2022] GET /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd => generated 298 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:25.455203 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528de15-2bf8-4667-a668-d5e34e9947d5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:25.475459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 187/388] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:25.498654 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3b440-d77e-4f42-b2a6-94a76e41a10d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:25.622121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f3b440-d77e-4f42-b2a6-94a76e41a10d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:25.665919 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282d2b7-1424-461b-a2ea-a875169b15d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:25.668449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282d2b7-1424-461b-a2ea-a875169b15d2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:25.670239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282d2b7-1424-461b-a2ea-a875169b15d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:25.670770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c282d2b7-1424-461b-a2ea-a875169b15d2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:25.695623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c282d2b7-1424-461b-a2ea-a875169b15d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:25.705578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-c282d2b7-1424-461b-a2ea-a875169b15d2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:53:25.705578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:25.705578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:25.705578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:25.705578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:25.705578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:25.705578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:25.705578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:25.705578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:25.705578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:25.705578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:25.705578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:25.705578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:25.705578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:25.705578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:25.705578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:25.706485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:25.706485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:25.706485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:25.706485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:25.706485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:25.706485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:25.706485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:25.706485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:25.706485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:25.706485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:25.706485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:25.706485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:25.706485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:25.706485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:25.706485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:25.706485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:25.707280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:25.707280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:25.707280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:25.707280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:25.707280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:25.707280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:25.707280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:25.707280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:25.707280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:25.707280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:25.707280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:53:25.707280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:25.709865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 188/389] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:53:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:53:25.714277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3528de15-2bf8-4667-a668-d5e34e9947d5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:25.715515 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282d2b7-1424-461b-a2ea-a875169b15d2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:25.747666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:25.798088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c364da9-1772-43dc-a894-53ce44eed697 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:25.799107 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c364da9-1772-43dc-a894-53ce44eed697 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:25.799395 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c364da9-1772-43dc-a894-53ce44eed697 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:25.799674 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c364da9-1772-43dc-a894-53ce44eed697 None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd/groups/1f6dd111ebd740e1a7ae97ca6211ffd0/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:25.804875 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c364da9-1772-43dc-a894-53ce44eed697 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c32ce113f734d9cbd5e90555ab94cdd, group_id=1f6dd111ebd740e1a7ae97ca6211ffd0, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:25.806608 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c364da9-1772-43dc-a894-53ce44eed697 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:25.822853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 202/390] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Jul 11 13:53:25 2022] PUT /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd/groups/1f6dd111ebd740e1a7ae97ca6211ffd0/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:53:25.873813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c282d2b7-1424-461b-a2ea-a875169b15d2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:25.913517 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56e134-25bf-4444-989a-3d408f393f68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:25.914335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56e134-25bf-4444-989a-3d408f393f68 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:25.914588 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56e134-25bf-4444-989a-3d408f393f68 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:25.914868 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef56e134-25bf-4444-989a-3d408f393f68 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:25.915674 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56e134-25bf-4444-989a-3d408f393f68 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:25.915986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56e134-25bf-4444-989a-3d408f393f68 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:25.917425 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef56e134-25bf-4444-989a-3d408f393f68 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:25.927456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 189/391] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:25 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:25.933304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c364da9-1772-43dc-a894-53ce44eed697 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:25.976140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:26.060781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c364da9-1772-43dc-a894-53ce44eed697 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:26.100436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514536a7-9407-4fcb-b383-d6d7a6fdaba6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:26.101302 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514536a7-9407-4fcb-b383-d6d7a6fdaba6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:26.101590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514536a7-9407-4fcb-b383-d6d7a6fdaba6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:26.101869 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514536a7-9407-4fcb-b383-d6d7a6fdaba6 None admin] PATH_INFO: `/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:26.103118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514536a7-9407-4fcb-b383-d6d7a6fdaba6 None admin] RBAC: Authorizing `identity:get_group(group_id=1f6dd111ebd740e1a7ae97ca6211ffd0)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:26.104554 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514536a7-9407-4fcb-b383-d6d7a6fdaba6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:26.106495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 203/392] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 11 13:53:25 2022] GET /identity/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0 => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:26.112120 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56e134-25bf-4444-989a-3d408f393f68 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:26.195544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:26.221804 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef56e134-25bf-4444-989a-3d408f393f68 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:26.256484 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f65da4df-b34b-462f-bf10-b3d1b78d8bf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:26.257413 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65da4df-b34b-462f-bf10-b3d1b78d8bf6 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:26.257698 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65da4df-b34b-462f-bf10-b3d1b78d8bf6 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:26.257977 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65da4df-b34b-462f-bf10-b3d1b78d8bf6 None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:26.259284 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65da4df-b34b-462f-bf10-b3d1b78d8bf6 None admin] RBAC: Authorizing `identity:get_project(project_id=9c32ce113f734d9cbd5e90555ab94cdd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:26.260784 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f65da4df-b34b-462f-bf10-b3d1b78d8bf6 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:26.262804 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 190/393] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:26 2022] GET /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd => generated 298 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:26.269231 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514536a7-9407-4fcb-b383-d6d7a6fdaba6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:26.319375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e7feb-817a-4b57-b092-43225e498ff6 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:26.319665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e7feb-817a-4b57-b092-43225e498ff6 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:26.319917 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-246e7feb-817a-4b57-b092-43225e498ff6 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:26.321573 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 191/394] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:53:26.331704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36a2de-cb52-4e08-9ee8-5a9f1cce91af None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:26.332147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36a2de-cb52-4e08-9ee8-5a9f1cce91af None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:26.332531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac36a2de-cb52-4e08-9ee8-5a9f1cce91af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:26.359400 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ac36a2de-cb52-4e08-9ee8-5a9f1cce91af None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:26.361971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ac36a2de-cb52-4e08-9ee8-5a9f1cce91af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:26.421852 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514536a7-9407-4fcb-b383-d6d7a6fdaba6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:26.427413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:26.473000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 192/395] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:26.496256 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ba029-3281-45a7-8815-67ef889abd00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:26.496744 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36a2de-cb52-4e08-9ee8-5a9f1cce91af None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:26.497162 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ba029-3281-45a7-8815-67ef889abd00 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:26.497445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ba029-3281-45a7-8815-67ef889abd00 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:26.497725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ba029-3281-45a7-8815-67ef889abd00 None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd/groups/1f6dd111ebd740e1a7ae97ca6211ffd0/roles/37f000302df54045892d35f57104660a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:26.500546 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ba029-3281-45a7-8815-67ef889abd00 None admin] RBAC: Authorizing `identity:create_grant(project_id=9c32ce113f734d9cbd5e90555ab94cdd, group_id=1f6dd111ebd740e1a7ae97ca6211ffd0, role_id=37f000302df54045892d35f57104660a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:26.501989 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ba029-3281-45a7-8815-67ef889abd00 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:26.530960 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 204/396] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Jul 11 13:53:26 2022] PUT /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd/groups/1f6dd111ebd740e1a7ae97ca6211ffd0/roles/37f000302df54045892d35f57104660a => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:53:26.596824 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:26.630480 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac36a2de-cb52-4e08-9ee8-5a9f1cce91af None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:26.667795 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646370a-4521-4acc-8156-02d90223334c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:26.668677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646370a-4521-4acc-8156-02d90223334c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:26.668965 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646370a-4521-4acc-8156-02d90223334c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:26.669236 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e646370a-4521-4acc-8156-02d90223334c None admin] PATH_INFO: `/v3/roles/member` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:26.681942 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e646370a-4521-4acc-8156-02d90223334c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:26.690791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-e646370a-4521-4acc-8156-02d90223334c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:26.690791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:26.690791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:26.690791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:26.690791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:26.690791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:26.690791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:26.690791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:26.690791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:26.690791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:26.690791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:26.690791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:26.690791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:26.690791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:26.690791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:26.690791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:26.691737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:26.691737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:26.691737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:26.691737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:26.691737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:26.691737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:26.691737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:26.691737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:26.691737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:26.691737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:26.691737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:26.691737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:26.691737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:26.691737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:26.691737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:26.691737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:26.692522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:26.692522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:26.692522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:26.692522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:26.692522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:26.692522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:26.692522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:26.692522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:26.692522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:26.692522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:26.692522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:26.692522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:26.692522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 193/397] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:53:26.699927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ba029-3281-45a7-8815-67ef889abd00 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:26.816405 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ba029-3281-45a7-8815-67ef889abd00 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:26.852255 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109287f7-c704-42d0-90d3-577e67fa59b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:26.852858 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109287f7-c704-42d0-90d3-577e67fa59b2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:26.853148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109287f7-c704-42d0-90d3-577e67fa59b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:26.853428 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-109287f7-c704-42d0-90d3-577e67fa59b2 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:26.854192 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109287f7-c704-42d0-90d3-577e67fa59b2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:26.854561 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109287f7-c704-42d0-90d3-577e67fa59b2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:26.855938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-109287f7-c704-42d0-90d3-577e67fa59b2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:26.869129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 205/398] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:53:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:26.875404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646370a-4521-4acc-8156-02d90223334c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:26.991147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e646370a-4521-4acc-8156-02d90223334c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:27.028671 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:27.039110 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06373525-5d2e-4fc1-87bf-87a388f18df1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:27.040310 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06373525-5d2e-4fc1-87bf-87a388f18df1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:27.040798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06373525-5d2e-4fc1-87bf-87a388f18df1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:27.041296 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06373525-5d2e-4fc1-87bf-87a388f18df1 None admin] PATH_INFO: `/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:27.042801 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06373525-5d2e-4fc1-87bf-87a388f18df1 None admin] RBAC: Authorizing `identity:get_group(group_id=1f6dd111ebd740e1a7ae97ca6211ffd0)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:27.044491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06373525-5d2e-4fc1-87bf-87a388f18df1 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:27.047373 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 194/399] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 11 13:53:26 2022] GET /identity/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0 => generated 226 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:27.052667 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109287f7-c704-42d0-90d3-577e67fa59b2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:27.098931 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91216151-7380-4094-b337-8404171a31fe None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:27.099566 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91216151-7380-4094-b337-8404171a31fe None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:27.100069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91216151-7380-4094-b337-8404171a31fe None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:27.101882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 195/400] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:27 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:27.105702 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:27.107681 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ce011-f884-4f87-bdb0-a5cb30a4bc94 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:27.108232 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ce011-f884-4f87-bdb0-a5cb30a4bc94 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:27.108831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-746ce011-f884-4f87-bdb0-a5cb30a4bc94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:27.126973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-746ce011-f884-4f87-bdb0-a5cb30a4bc94 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:27.128501 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-746ce011-f884-4f87-bdb0-a5cb30a4bc94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:27.186350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-109287f7-c704-42d0-90d3-577e67fa59b2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:27.219965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162b7d6-73cf-4008-b9f9-610fdcb3774b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:27.220837 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162b7d6-73cf-4008-b9f9-610fdcb3774b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:27.221117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162b7d6-73cf-4008-b9f9-610fdcb3774b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:27.221392 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162b7d6-73cf-4008-b9f9-610fdcb3774b None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:27.222635 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162b7d6-73cf-4008-b9f9-610fdcb3774b None admin] RBAC: Authorizing `identity:get_project(project_id=34304bd3ae654c1c976f55ca788ae592)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:27.224072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162b7d6-73cf-4008-b9f9-610fdcb3774b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:27.226073 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 206/401] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:27 2022] GET /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:27.232195 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162b7d6-73cf-4008-b9f9-610fdcb3774b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:27.240828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 196/402] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:53:27.263279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ce011-f884-4f87-bdb0-a5cb30a4bc94 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:27.351190 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:27.360049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162b7d6-73cf-4008-b9f9-610fdcb3774b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:27.411048 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9db59-1217-4833-aa7b-6080e1df21a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:27.412177 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9db59-1217-4833-aa7b-6080e1df21a3 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:27.412607 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9db59-1217-4833-aa7b-6080e1df21a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:27.413026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f9db59-1217-4833-aa7b-6080e1df21a3 None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592/groups/1f6dd111ebd740e1a7ae97ca6211ffd0/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:27.416094 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9db59-1217-4833-aa7b-6080e1df21a3 None admin] RBAC: Authorizing `identity:create_grant(project_id=34304bd3ae654c1c976f55ca788ae592, group_id=1f6dd111ebd740e1a7ae97ca6211ffd0, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:27.418878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f9db59-1217-4833-aa7b-6080e1df21a3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:27.437149 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 207/403] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Jul 11 13:53:27 2022] PUT /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592/groups/1f6dd111ebd740e1a7ae97ca6211ffd0/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:53:27.486635 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-746ce011-f884-4f87-bdb0-a5cb30a4bc94 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:27.533772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:27.542875 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63822180-d0eb-49fc-b3df-3943b85d0de2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:27.544010 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63822180-d0eb-49fc-b3df-3943b85d0de2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:27.544470 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63822180-d0eb-49fc-b3df-3943b85d0de2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:27.544920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63822180-d0eb-49fc-b3df-3943b85d0de2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:27.558420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63822180-d0eb-49fc-b3df-3943b85d0de2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:27.567153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-63822180-d0eb-49fc-b3df-3943b85d0de2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:53:27.567153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:27.567153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:27.567153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:27.567153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:27.567153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:27.567153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:27.567153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:27.567153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:27.567153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:27.567153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:27.567153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:27.567153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:27.567153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:27.567153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:27.567153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:27.568109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:27.568109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:27.568109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:27.568109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:27.568109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:27.568109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:27.568109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:27.568109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:27.568109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:27.568109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:27.568109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:27.568109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:27.568109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:27.568109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:27.568109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:27.568109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:27.568879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:27.568879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:27.568879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:27.568879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:27.568879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:27.568879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:27.568879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:27.568879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:27.568879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:27.568879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:27.568879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:53:27.568879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:27.571309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 197/404] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:53:27 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 308 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:53:27.579747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9db59-1217-4833-aa7b-6080e1df21a3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:27.674992 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02adf36-c3dd-4bff-96c6-03e4ffecc545 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:27.675378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02adf36-c3dd-4bff-96c6-03e4ffecc545 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:27.675741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02adf36-c3dd-4bff-96c6-03e4ffecc545 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:27.677510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 198/405] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:27.688319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c60117-bd36-48db-ad7a-94f898f57aeb None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:27.688613 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c60117-bd36-48db-ad7a-94f898f57aeb None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:27.688898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c60117-bd36-48db-ad7a-94f898f57aeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:27.715250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f9db59-1217-4833-aa7b-6080e1df21a3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:27.718581 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-96c60117-bd36-48db-ad7a-94f898f57aeb None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:27.719817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-96c60117-bd36-48db-ad7a-94f898f57aeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:27.767591 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1373f03-3257-47f8-8c39-d1ba44ede33c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:27.768733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1373f03-3257-47f8-8c39-d1ba44ede33c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:27.769026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1373f03-3257-47f8-8c39-d1ba44ede33c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:27.769311 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1373f03-3257-47f8-8c39-d1ba44ede33c None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:27.770518 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1373f03-3257-47f8-8c39-d1ba44ede33c None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:27.771105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1373f03-3257-47f8-8c39-d1ba44ede33c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:27.772751 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1373f03-3257-47f8-8c39-d1ba44ede33c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:27.783795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 208/406] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:27 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:27.789618 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1373f03-3257-47f8-8c39-d1ba44ede33c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:27.801496 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 199/407] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:27.811374 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:27.826561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c60117-bd36-48db-ad7a-94f898f57aeb None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:27.918312 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1373f03-3257-47f8-8c39-d1ba44ede33c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:27.971916 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96c60117-bd36-48db-ad7a-94f898f57aeb None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:27.973874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c25ccd-21c0-43ad-a4d7-31b7297f61f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:27.974775 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c25ccd-21c0-43ad-a4d7-31b7297f61f6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:27.975064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c25ccd-21c0-43ad-a4d7-31b7297f61f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:27.975340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68c25ccd-21c0-43ad-a4d7-31b7297f61f6 None admin] PATH_INFO: `/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:27.977278 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c25ccd-21c0-43ad-a4d7-31b7297f61f6 None admin] RBAC: Authorizing `identity:get_group(group_id=1f6dd111ebd740e1a7ae97ca6211ffd0)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:27.982175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68c25ccd-21c0-43ad-a4d7-31b7297f61f6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:27.985679 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 209/408] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 11 13:53:27 2022] GET /identity/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:27.992150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c25ccd-21c0-43ad-a4d7-31b7297f61f6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:28.020966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:28.053996 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4e475-6d5c-47b7-bfa8-844c07e1b359 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:28.057042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4e475-6d5c-47b7-bfa8-844c07e1b359 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:28.057629 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4e475-6d5c-47b7-bfa8-844c07e1b359 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:28.058525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a4e475-6d5c-47b7-bfa8-844c07e1b359 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:28.083074 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a4e475-6d5c-47b7-bfa8-844c07e1b359 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:28.093070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-a3a4e475-6d5c-47b7-bfa8-844c07e1b359 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:53:28.093070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:28.093070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:28.093070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:28.093070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:28.093070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:28.093070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:28.093070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:28.093070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:28.093070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:28.093070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:28.093070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:28.093070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:28.093070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:28.093070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:28.093070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:28.094067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:28.094067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:28.094067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:28.094067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:28.094067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:28.094067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:28.094067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:28.094067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:28.094067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:28.094067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:28.094067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:28.094067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:28.094067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:28.094067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:28.094067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:28.094067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:28.094912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:28.094912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:28.094912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:28.094912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:28.094912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:28.094912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:28.094912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:28.094912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:28.094912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:28.094912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:28.094912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:53:28.094912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:28.094912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 200/409] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:53:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:53:28.111331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4e475-6d5c-47b7-bfa8-844c07e1b359 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:28.138730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:28.162234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68c25ccd-21c0-43ad-a4d7-31b7297f61f6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:28.209106 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b224c-024c-4c29-b8d8-63066738862a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:28.210000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b224c-024c-4c29-b8d8-63066738862a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:28.210281 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b224c-024c-4c29-b8d8-63066738862a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:28.210580 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b82b224c-024c-4c29-b8d8-63066738862a None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:28.212234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b224c-024c-4c29-b8d8-63066738862a None admin] RBAC: Authorizing `identity:get_project(project_id=34304bd3ae654c1c976f55ca788ae592)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:28.213681 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b82b224c-024c-4c29-b8d8-63066738862a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:28.215782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 210/410] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:27 2022] GET /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592 => generated 302 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:28.222779 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b224c-024c-4c29-b8d8-63066738862a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:28.293537 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a4e475-6d5c-47b7-bfa8-844c07e1b359 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:28.337998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6048d-ac2d-4fd1-93cf-68910886f413 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:28.339148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6048d-ac2d-4fd1-93cf-68910886f413 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:28.339632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6048d-ac2d-4fd1-93cf-68910886f413 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:28.340114 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58e6048d-ac2d-4fd1-93cf-68910886f413 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:28.341394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6048d-ac2d-4fd1-93cf-68910886f413 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:28.341874 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6048d-ac2d-4fd1-93cf-68910886f413 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:28.344948 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58e6048d-ac2d-4fd1-93cf-68910886f413 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:28.356320 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 201/411] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:28 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:28.362356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6048d-ac2d-4fd1-93cf-68910886f413 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:28.411544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:28.475823 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b82b224c-024c-4c29-b8d8-63066738862a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:28.478234 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58e6048d-ac2d-4fd1-93cf-68910886f413 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:28.534624 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b49b64-9a72-4ea3-b197-348eb5c75abc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:28.536000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b49b64-9a72-4ea3-b197-348eb5c75abc None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:28.537587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b49b64-9a72-4ea3-b197-348eb5c75abc None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:28.538702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1b49b64-9a72-4ea3-b197-348eb5c75abc None admin] PATH_INFO: `/v3/groups/701f37c2b4504b3abe423ce570d8f48e` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:28.540872 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b49b64-9a72-4ea3-b197-348eb5c75abc None admin] RBAC: Authorizing `identity:get_group(group_id=701f37c2b4504b3abe423ce570d8f48e)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:28.542718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1b49b64-9a72-4ea3-b197-348eb5c75abc None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:28.546131 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 202/412] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 11 13:53:28 2022] GET /identity/v3/groups/701f37c2b4504b3abe423ce570d8f48e => generated 229 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:28.552229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b49b64-9a72-4ea3-b197-348eb5c75abc None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:28.554475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337416f1-f7e5-43d3-90c4-df8c3d2b3834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:28.555636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337416f1-f7e5-43d3-90c4-df8c3d2b3834 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:28.555922 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337416f1-f7e5-43d3-90c4-df8c3d2b3834 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:28.556201 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337416f1-f7e5-43d3-90c4-df8c3d2b3834 None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592/groups/1f6dd111ebd740e1a7ae97ca6211ffd0/roles/37f000302df54045892d35f57104660a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:28.561498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337416f1-f7e5-43d3-90c4-df8c3d2b3834 None admin] RBAC: Authorizing `identity:create_grant(project_id=34304bd3ae654c1c976f55ca788ae592, group_id=1f6dd111ebd740e1a7ae97ca6211ffd0, role_id=37f000302df54045892d35f57104660a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:28.567874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-337416f1-f7e5-43d3-90c4-df8c3d2b3834 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:28.586003 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 211/413] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Jul 11 13:53:28 2022] PUT /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592/groups/1f6dd111ebd740e1a7ae97ca6211ffd0/roles/37f000302df54045892d35f57104660a => generated 0 bytes in 364 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:53:28.613263 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce55ce-c837-4637-a13e-28738ee24f29 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:28.613587 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce55ce-c837-4637-a13e-28738ee24f29 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:28.613833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dce55ce-c837-4637-a13e-28738ee24f29 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:28.615672 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 212/414] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:28.624521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70da220-4677-4751-819a-a9cc209057db None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:28.624889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70da220-4677-4751-819a-a9cc209057db None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:28.625127 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b70da220-4677-4751-819a-a9cc209057db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:28.672075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b70da220-4677-4751-819a-a9cc209057db None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:28.673384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b70da220-4677-4751-819a-a9cc209057db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:28.685758 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:28.709709 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1b49b64-9a72-4ea3-b197-348eb5c75abc None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:28.726585 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:28.787359 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6965f6-3be2-45c1-ad1a-d0866dcb0be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:28.788353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6965f6-3be2-45c1-ad1a-d0866dcb0be0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:28.788648 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6965f6-3be2-45c1-ad1a-d0866dcb0be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:28.788926 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af6965f6-3be2-45c1-ad1a-d0866dcb0be0 None admin] PATH_INFO: `/v3/projects/5898aef39ca74b7d8b99cb1aa700e9c2` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:28.791036 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6965f6-3be2-45c1-ad1a-d0866dcb0be0 None admin] RBAC: Authorizing `identity:get_project(project_id=5898aef39ca74b7d8b99cb1aa700e9c2)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:28.792490 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af6965f6-3be2-45c1-ad1a-d0866dcb0be0 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:28.794614 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 203/415] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:28 2022] GET /identity/v3/projects/5898aef39ca74b7d8b99cb1aa700e9c2 => generated 344 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:28.799969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 213/416] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:53:28.800759 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6965f6-3be2-45c1-ad1a-d0866dcb0be0 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:28.823827 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70da220-4677-4751-819a-a9cc209057db None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:28.940374 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b70da220-4677-4751-819a-a9cc209057db None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:28.979805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac7501-e552-4365-8d30-65ab1bad5673 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:28.980534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac7501-e552-4365-8d30-65ab1bad5673 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:28.980821 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac7501-e552-4365-8d30-65ab1bad5673 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:28.981099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ac7501-e552-4365-8d30-65ab1bad5673 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:28.993324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ac7501-e552-4365-8d30-65ab1bad5673 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:29.002230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b4ac7501-e552-4365-8d30-65ab1bad5673 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:29.002230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:29.002230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:29.002230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:29.002230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:29.002230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:29.002230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:29.002230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:29.002230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:29.002230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:29.002230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:29.002230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:29.002230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:29.002230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:29.002230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:29.002230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:29.003144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:29.003144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:29.003144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:29.003144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:29.003144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:29.003144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:29.003144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:29.003144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:29.003144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:29.003144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:29.003144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:29.003144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:29.003144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:29.003144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:29.003144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:29.003144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:29.004033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:29.004033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:29.004033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:29.004033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:29.004033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:29.004033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:29.004033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:29.004033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:29.004033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:29.004033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:29.004033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:29.004033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:29.004033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 214/417] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:53:29.009475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac7501-e552-4365-8d30-65ab1bad5673 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:29.014641 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:29.100411 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af6965f6-3be2-45c1-ad1a-d0866dcb0be0 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:29.129528 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ac7501-e552-4365-8d30-65ab1bad5673 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:29.170022 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec882ba-cf5b-47bf-a7a1-ee04c22300c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:29.170788 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec882ba-cf5b-47bf-a7a1-ee04c22300c3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:29.171018 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec882ba-cf5b-47bf-a7a1-ee04c22300c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:29.171301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec882ba-cf5b-47bf-a7a1-ee04c22300c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:29.172105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec882ba-cf5b-47bf-a7a1-ee04c22300c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:29.172416 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec882ba-cf5b-47bf-a7a1-ee04c22300c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:29.173784 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec882ba-cf5b-47bf-a7a1-ee04c22300c3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:29.175729 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a9f606-f559-4d03-8872-512f53c515cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:29.176734 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9f606-f559-4d03-8872-512f53c515cb None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:29.177025 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9f606-f559-4d03-8872-512f53c515cb None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:29.177307 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a9f606-f559-4d03-8872-512f53c515cb None admin] PATH_INFO: `/v3/projects/5898aef39ca74b7d8b99cb1aa700e9c2/groups/701f37c2b4504b3abe423ce570d8f48e/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:29.183474 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 215/418] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:53:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:29.183770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9f606-f559-4d03-8872-512f53c515cb None admin] RBAC: Authorizing `identity:create_grant(project_id=5898aef39ca74b7d8b99cb1aa700e9c2, group_id=701f37c2b4504b3abe423ce570d8f48e, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:29.185233 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a9f606-f559-4d03-8872-512f53c515cb None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:29.190971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec882ba-cf5b-47bf-a7a1-ee04c22300c3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:29.207694 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 204/419] 10.222.0.49 () {58 vars in 1442 bytes} [Mon Jul 11 13:53:28 2022] PUT /identity/v3/projects/5898aef39ca74b7d8b99cb1aa700e9c2/groups/701f37c2b4504b3abe423ce570d8f48e/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:53:29.260552 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:29.301337 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec882ba-cf5b-47bf-a7a1-ee04c22300c3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:29.339692 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32505bf-b23f-4773-a67a-a632af62b999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:29.340360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32505bf-b23f-4773-a67a-a632af62b999 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:29.340654 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32505bf-b23f-4773-a67a-a632af62b999 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:29.340987 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32505bf-b23f-4773-a67a-a632af62b999 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:29.343137 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf349f-bc9c-48ab-8412-5a0600b1f9f5 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:29.343474 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf349f-bc9c-48ab-8412-5a0600b1f9f5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:29.343875 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8cf349f-bc9c-48ab-8412-5a0600b1f9f5 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:29.345722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 205/420] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:29.353750 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521a6e9-7a37-45ea-ae73-f81a0cd5098a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:29.354043 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521a6e9-7a37-45ea-ae73-f81a0cd5098a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:29.354330 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5521a6e9-7a37-45ea-ae73-f81a0cd5098a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:29.386722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5521a6e9-7a37-45ea-ae73-f81a0cd5098a None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:29.388093 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5521a6e9-7a37-45ea-ae73-f81a0cd5098a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:29.423098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32505bf-b23f-4773-a67a-a632af62b999 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:29.424586 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32505bf-b23f-4773-a67a-a632af62b999 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:29.449296 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:29.489826 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 206/421] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:53:29.516977 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521a6e9-7a37-45ea-ae73-f81a0cd5098a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:29.518747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:29.547537 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 216/422] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:29 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 13:53:29.559385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32505bf-b23f-4773-a67a-a632af62b999 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:29.704127 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32505bf-b23f-4773-a67a-a632af62b999 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:29.705954 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5521a6e9-7a37-45ea-ae73-f81a0cd5098a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:29.749783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da5bb1e-1684-4698-a4fd-a50b3a423614 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:29.750682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5bb1e-1684-4698-a4fd-a50b3a423614 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:29.750962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5bb1e-1684-4698-a4fd-a50b3a423614 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:29.751233 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6da5bb1e-1684-4698-a4fd-a50b3a423614 None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:29.753607 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da5bb1e-1684-4698-a4fd-a50b3a423614 None admin] RBAC: Authorizing `identity:get_project(project_id=9c32ce113f734d9cbd5e90555ab94cdd)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:29.755085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6da5bb1e-1684-4698-a4fd-a50b3a423614 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:29.756951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 217/423] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:29 2022] GET /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd => generated 298 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:29.762614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da5bb1e-1684-4698-a4fd-a50b3a423614 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:29.789469 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8528a-6053-49e1-82a6-5ba7f52deed1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:29.790203 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8528a-6053-49e1-82a6-5ba7f52deed1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:29.790506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8528a-6053-49e1-82a6-5ba7f52deed1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:29.790778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ff8528a-6053-49e1-82a6-5ba7f52deed1 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:29.811952 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ff8528a-6053-49e1-82a6-5ba7f52deed1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:29.821802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-9ff8528a-6053-49e1-82a6-5ba7f52deed1 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:53:29.821802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:29.821802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:29.821802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:29.821802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:29.821802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:29.821802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:29.821802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:29.821802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:29.821802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:29.821802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:29.821802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:29.821802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:29.821802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:29.821802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:29.821802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:29.822722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:29.822722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:29.822722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:29.822722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:29.822722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:29.822722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:29.822722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:29.822722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:29.822722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:29.822722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:29.822722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:29.822722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:29.822722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:29.822722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:29.822722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:29.822722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:29.823544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:29.823544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:29.823544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:29.823544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:29.823544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:29.823544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:29.823544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:29.823544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:29.823544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:29.823544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:29.823544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:53:29.823544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:29.827361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:29.839254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 207/424] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:53:29 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 323 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:29.849203 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8528a-6053-49e1-82a6-5ba7f52deed1 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:29.900201 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:29.929651 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6da5bb1e-1684-4698-a4fd-a50b3a423614 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:29.976830 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a5ab89-d1f1-4175-a3ce-a2e0231d0b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:29.978059 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5ab89-d1f1-4175-a3ce-a2e0231d0b96 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:29.978573 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5ab89-d1f1-4175-a3ce-a2e0231d0b96 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:29.979584 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a5ab89-d1f1-4175-a3ce-a2e0231d0b96 None admin] PATH_INFO: `/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:29.981214 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a5ab89-d1f1-4175-a3ce-a2e0231d0b96 None admin] RBAC: Authorizing `identity:get_group(group_id=1f6dd111ebd740e1a7ae97ca6211ffd0)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:29.983738 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a5ab89-d1f1-4175-a3ce-a2e0231d0b96 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:29.986688 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 218/425] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 11 13:53:29 2022] GET /identity/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0 => generated 226 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:29.988397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61293098-00cd-43a2-92bb-5960574aa3f5 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:29.988986 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61293098-00cd-43a2-92bb-5960574aa3f5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:29.989389 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61293098-00cd-43a2-92bb-5960574aa3f5 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:29.990847 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 219/426] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:29.992448 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61293098-00cd-43a2-92bb-5960574aa3f5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:29.998213 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ff8528a-6053-49e1-82a6-5ba7f52deed1 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:30.040144 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1997402-1e54-4ac0-a906-b80a884a6f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:30.040676 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1997402-1e54-4ac0-a906-b80a884a6f35 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:30.040967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1997402-1e54-4ac0-a906-b80a884a6f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:30.041245 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1997402-1e54-4ac0-a906-b80a884a6f35 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:30.042018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1997402-1e54-4ac0-a906-b80a884a6f35 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:30.042352 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1997402-1e54-4ac0-a906-b80a884a6f35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:30.043801 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1997402-1e54-4ac0-a906-b80a884a6f35 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:30.059168 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 208/427] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:29 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:30.065396 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dac259-5a2e-4746-8baf-c2a91c7aefbb None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:30.065396 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dac259-5a2e-4746-8baf-c2a91c7aefbb None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:30.065396 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3dac259-5a2e-4746-8baf-c2a91c7aefbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:30.083358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c3dac259-5a2e-4746-8baf-c2a91c7aefbb None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:30.086388 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c3dac259-5a2e-4746-8baf-c2a91c7aefbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:30.126527 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:30.132088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61293098-00cd-43a2-92bb-5960574aa3f5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:30.169271 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5ab2e-eb16-4d47-a976-a14bdef9e3cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:30.170273 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5ab2e-eb16-4d47-a976-a14bdef9e3cf None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:30.171189 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5ab2e-eb16-4d47-a976-a14bdef9e3cf None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:30.171750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e5ab2e-eb16-4d47-a976-a14bdef9e3cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:30.172819 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5ab2e-eb16-4d47-a976-a14bdef9e3cf None admin] RBAC: Adding query filter params (group.id=1f6dd111ebd740e1a7ae97ca6211ffd0, role.id=827f1f1239754bf797e64f270ba1a80c, scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:30.173259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 209/428] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:30.173860 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5ab2e-eb16-4d47-a976-a14bdef9e3cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:30.174100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dac259-5a2e-4746-8baf-c2a91c7aefbb None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:30.175904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e5ab2e-eb16-4d47-a976-a14bdef9e3cf None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:30.183238 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 220/429] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 11 13:53:29 2022] GET /identity/v3/role_assignments?group.id=1f6dd111ebd740e1a7ae97ca6211ffd0&scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd&role.id=827f1f1239754bf797e64f270ba1a80c => generated 611 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:30.198003 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:30.224680 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5ab2e-eb16-4d47-a976-a14bdef9e3cf None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:30.293677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3dac259-5a2e-4746-8baf-c2a91c7aefbb None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:30.337191 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ae1a7-90b3-400f-a9b4-5b68b98f095a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:30.337865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ae1a7-90b3-400f-a9b4-5b68b98f095a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:30.338151 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ae1a7-90b3-400f-a9b4-5b68b98f095a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:30.338460 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586ae1a7-90b3-400f-a9b4-5b68b98f095a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:30.395000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e5ab2e-eb16-4d47-a976-a14bdef9e3cf None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:30.422366 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ae1a7-90b3-400f-a9b4-5b68b98f095a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:30.423981 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586ae1a7-90b3-400f-a9b4-5b68b98f095a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:30.437787 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16979ab5-04ec-43f8-a7e4-85be6f946269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:30.439090 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16979ab5-04ec-43f8-a7e4-85be6f946269 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:30.439599 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16979ab5-04ec-43f8-a7e4-85be6f946269 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:30.440077 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16979ab5-04ec-43f8-a7e4-85be6f946269 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:30.449907 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:30.458072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16979ab5-04ec-43f8-a7e4-85be6f946269 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:30.467222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-16979ab5-04ec-43f8-a7e4-85be6f946269 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:30.467222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:30.467222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:30.467222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:30.467222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:30.467222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:30.467222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:30.467222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:30.467222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:30.467222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:30.467222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:30.467222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:30.467222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:30.467222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:30.467222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:30.467222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:30.468117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:30.468117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:30.468117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:30.468117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:30.468117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:30.468117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:30.468117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:30.468117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:30.468117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:30.468117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:30.468117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:30.468117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:30.468117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:30.468117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:30.468117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:30.468117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:30.468969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:30.468969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:30.468969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:30.468969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:30.468969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:30.468969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:30.468969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:30.468969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:30.468969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:30.468969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:30.468969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 11 13:53:30.468969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:30.471802 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 221/430] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:53:30.479384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16979ab5-04ec-43f8-a7e4-85be6f946269 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:30.514206 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:30.540109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 210/431] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:30.546973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ae1a7-90b3-400f-a9b4-5b68b98f095a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:30.613952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16979ab5-04ec-43f8-a7e4-85be6f946269 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:30.653515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4637f09-4150-4030-90c3-f89d7448c331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:30.654190 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4637f09-4150-4030-90c3-f89d7448c331 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:30.654509 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4637f09-4150-4030-90c3-f89d7448c331 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:30.654791 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4637f09-4150-4030-90c3-f89d7448c331 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:30.655561 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4637f09-4150-4030-90c3-f89d7448c331 None admin] RBAC: Adding query filter params (name=member) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:30.656417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4637f09-4150-4030-90c3-f89d7448c331 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:30.657805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4637f09-4150-4030-90c3-f89d7448c331 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:30.667975 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 222/432] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:53:30 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:30.674667 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4637f09-4150-4030-90c3-f89d7448c331 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:30.684588 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586ae1a7-90b3-400f-a9b4-5b68b98f095a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:30.721230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:30.732282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183b7f1-5c93-43e2-8224-7ee408cc7d69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:30.733251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183b7f1-5c93-43e2-8224-7ee408cc7d69 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:30.733706 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183b7f1-5c93-43e2-8224-7ee408cc7d69 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:30.734636 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6183b7f1-5c93-43e2-8224-7ee408cc7d69 None admin] PATH_INFO: `/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:30.736302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183b7f1-5c93-43e2-8224-7ee408cc7d69 None admin] RBAC: Authorizing `identity:get_project(project_id=9c32ce113f734d9cbd5e90555ab94cdd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:30.738039 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6183b7f1-5c93-43e2-8224-7ee408cc7d69 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:30.743454 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 211/433] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:30 2022] GET /identity/v3/projects/9c32ce113f734d9cbd5e90555ab94cdd => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:30.746239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183b7f1-5c93-43e2-8224-7ee408cc7d69 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:30.824548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:30.844916 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4637f09-4150-4030-90c3-f89d7448c331 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:30.875513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6183b7f1-5c93-43e2-8224-7ee408cc7d69 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:30.891661 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e83002c-9520-47ab-bdb7-1ca638d7a0b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:30.892770 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83002c-9520-47ab-bdb7-1ca638d7a0b4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:30.893387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83002c-9520-47ab-bdb7-1ca638d7a0b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:30.893884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e83002c-9520-47ab-bdb7-1ca638d7a0b4 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:30.927606 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537583f-d3ad-4cec-ae15-2ebc82922281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:30.929121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537583f-d3ad-4cec-ae15-2ebc82922281 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:30.929763 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537583f-d3ad-4cec-ae15-2ebc82922281 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:30.930541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a537583f-d3ad-4cec-ae15-2ebc82922281 None admin] PATH_INFO: `/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:30.932501 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537583f-d3ad-4cec-ae15-2ebc82922281 None admin] RBAC: Authorizing `identity:get_group(group_id=1f6dd111ebd740e1a7ae97ca6211ffd0)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:30.934946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a537583f-d3ad-4cec-ae15-2ebc82922281 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:30.937771 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 212/434] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 11 13:53:30 2022] GET /identity/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:30.943643 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537583f-d3ad-4cec-ae15-2ebc82922281 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:30.981549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83002c-9520-47ab-bdb7-1ca638d7a0b4 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:30.983149 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e83002c-9520-47ab-bdb7-1ca638d7a0b4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:31.006832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:31.054571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a537583f-d3ad-4cec-ae15-2ebc82922281 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:31.079569 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:31.092467 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36e1ea-098f-4b1c-bd94-551c4d51ff07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:31.094023 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36e1ea-098f-4b1c-bd94-551c4d51ff07 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:31.094572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36e1ea-098f-4b1c-bd94-551c4d51ff07 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:31.095057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36e1ea-098f-4b1c-bd94-551c4d51ff07 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:31.096157 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36e1ea-098f-4b1c-bd94-551c4d51ff07 None admin] RBAC: Adding query filter params (group.id=1f6dd111ebd740e1a7ae97ca6211ffd0, role.id=37f000302df54045892d35f57104660a, scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:31.096721 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36e1ea-098f-4b1c-bd94-551c4d51ff07 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:31.098516 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be36e1ea-098f-4b1c-bd94-551c4d51ff07 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:31.098923 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 223/435] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 13:53:31.100260 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b9997-8ad8-4a1d-a0c0-80abd857f844 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:31.100549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b9997-8ad8-4a1d-a0c0-80abd857f844 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:31.100855 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960b9997-8ad8-4a1d-a0c0-80abd857f844 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:31.102126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 224/436] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:31.104629 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b9997-8ad8-4a1d-a0c0-80abd857f844 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:31.107384 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 213/437] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 11 13:53:30 2022] GET /identity/v3/role_assignments?group.id=1f6dd111ebd740e1a7ae97ca6211ffd0&scope.project.id=9c32ce113f734d9cbd5e90555ab94cdd&role.id=37f000302df54045892d35f57104660a => generated 611 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:31.108914 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bed617-0342-4a53-a9c2-7cccaa20b2ad None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:31.109596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bed617-0342-4a53-a9c2-7cccaa20b2ad None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:31.110047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45bed617-0342-4a53-a9c2-7cccaa20b2ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:31.132560 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-45bed617-0342-4a53-a9c2-7cccaa20b2ad None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:31.133795 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-45bed617-0342-4a53-a9c2-7cccaa20b2ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:31.217728 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 214/438] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:31.229968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960b9997-8ad8-4a1d-a0c0-80abd857f844 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:31.243506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bed617-0342-4a53-a9c2-7cccaa20b2ad None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:31.275020 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f3b41-9307-4dfc-9568-03063a97d5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:31.275989 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f3b41-9307-4dfc-9568-03063a97d5be None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:31.276284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f3b41-9307-4dfc-9568-03063a97d5be None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:31.276671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f48f3b41-9307-4dfc-9568-03063a97d5be None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:31.278027 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f3b41-9307-4dfc-9568-03063a97d5be None admin] RBAC: Authorizing `identity:get_project(project_id=34304bd3ae654c1c976f55ca788ae592)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:31.279589 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f48f3b41-9307-4dfc-9568-03063a97d5be None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:31.282663 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 225/439] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:31 2022] GET /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592 => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:31.291500 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f3b41-9307-4dfc-9568-03063a97d5be None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:31.298891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:31.361705 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:31.373476 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45bed617-0342-4a53-a9c2-7cccaa20b2ad None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:31.416392 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6739c288-1054-46f8-8cb4-fb4de51e4ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:31.417209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739c288-1054-46f8-8cb4-fb4de51e4ad8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:31.417488 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739c288-1054-46f8-8cb4-fb4de51e4ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:31.417763 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6739c288-1054-46f8-8cb4-fb4de51e4ad8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:31.428139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f48f3b41-9307-4dfc-9568-03063a97d5be None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:31.436059 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6739c288-1054-46f8-8cb4-fb4de51e4ad8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:31.444828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-6739c288-1054-46f8-8cb4-fb4de51e4ad8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:53:31.444828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:31.444828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:31.444828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:31.444828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:31.444828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:31.444828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:31.444828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:31.444828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:31.444828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:31.444828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:31.444828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:31.444828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:31.444828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:31.444828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:31.444828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:31.445720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:31.445720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:31.445720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:31.445720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:31.445720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:31.445720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:31.445720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:31.445720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:31.445720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:31.445720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:31.445720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:31.445720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:31.445720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:31.445720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:31.445720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:31.445720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:31.446534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:31.446534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:31.446534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:31.446534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:31.446534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:31.446534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:31.446534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:31.446534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:31.446534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:31.446534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:31.446534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:53:31.446534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:31.446534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 215/440] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:53:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:31.452338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6739c288-1054-46f8-8cb4-fb4de51e4ad8 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:31.479282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbc2de-ec2b-4dc8-bdc0-ce1f4cdab884 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:31.480155 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbc2de-ec2b-4dc8-bdc0-ce1f4cdab884 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:31.480493 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbc2de-ec2b-4dc8-bdc0-ce1f4cdab884 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:31.480773 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39dbc2de-ec2b-4dc8-bdc0-ce1f4cdab884 None admin] PATH_INFO: `/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:31.482559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dbc2de-ec2b-4dc8-bdc0-ce1f4cdab884 None admin] RBAC: Authorizing `identity:get_group(group_id=1f6dd111ebd740e1a7ae97ca6211ffd0)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:31.485456 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39dbc2de-ec2b-4dc8-bdc0-ce1f4cdab884 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:31.485975 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 226/441] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 11 13:53:31 2022] GET /identity/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0 => generated 226 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:31.492531 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbc2de-ec2b-4dc8-bdc0-ce1f4cdab884 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:31.583265 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6739c288-1054-46f8-8cb4-fb4de51e4ad8 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:31.601881 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:31.628827 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39dbc2de-ec2b-4dc8-bdc0-ce1f4cdab884 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:31.633451 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492fa85-0d1f-4331-9936-0f8a7b731f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:31.634167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492fa85-0d1f-4331-9936-0f8a7b731f79 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:31.635058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492fa85-0d1f-4331-9936-0f8a7b731f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:31.635347 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b492fa85-0d1f-4331-9936-0f8a7b731f79 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:31.636172 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b492fa85-0d1f-4331-9936-0f8a7b731f79 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:31.636484 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b492fa85-0d1f-4331-9936-0f8a7b731f79 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:31.638336 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b492fa85-0d1f-4331-9936-0f8a7b731f79 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:31.649072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 216/442] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:31.654539 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492fa85-0d1f-4331-9936-0f8a7b731f79 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:31.659034 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:31.672921 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837bff23-0231-4cd3-b5f8-9a5c55cf92ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:31.673586 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bff23-0231-4cd3-b5f8-9a5c55cf92ac None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:31.673874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bff23-0231-4cd3-b5f8-9a5c55cf92ac None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:31.674152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837bff23-0231-4cd3-b5f8-9a5c55cf92ac None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:31.675082 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837bff23-0231-4cd3-b5f8-9a5c55cf92ac None admin] RBAC: Adding query filter params (group.id=1f6dd111ebd740e1a7ae97ca6211ffd0, role.id=827f1f1239754bf797e64f270ba1a80c, scope.project.id=34304bd3ae654c1c976f55ca788ae592) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:31.675386 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837bff23-0231-4cd3-b5f8-9a5c55cf92ac None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:31.676828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837bff23-0231-4cd3-b5f8-9a5c55cf92ac None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:31.694330 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 227/443] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 11 13:53:31 2022] GET /identity/v3/role_assignments?group.id=1f6dd111ebd740e1a7ae97ca6211ffd0&scope.project.id=34304bd3ae654c1c976f55ca788ae592&role.id=827f1f1239754bf797e64f270ba1a80c => generated 611 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:31.707133 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e13f9-028f-436a-9f84-828713d4cca7 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:31.707454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e13f9-028f-436a-9f84-828713d4cca7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:31.707706 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21e13f9-028f-436a-9f84-828713d4cca7 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:31.709381 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 228/444] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:53:31.715100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c939f-1ca5-4326-ba8d-0aaca2bc58eb None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:31.715430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c939f-1ca5-4326-ba8d-0aaca2bc58eb None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:31.715667 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53c939f-1ca5-4326-ba8d-0aaca2bc58eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:31.734828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e53c939f-1ca5-4326-ba8d-0aaca2bc58eb None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:31.736348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e53c939f-1ca5-4326-ba8d-0aaca2bc58eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:31.785924 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b492fa85-0d1f-4331-9936-0f8a7b731f79 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:31.814401 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 229/445] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:31.823769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd50d22-6429-428d-b8e6-88b2a2334cfd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:31.824430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd50d22-6429-428d-b8e6-88b2a2334cfd None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:31.824717 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd50d22-6429-428d-b8e6-88b2a2334cfd None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:31.824996 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dd50d22-6429-428d-b8e6-88b2a2334cfd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:31.837391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c939f-1ca5-4326-ba8d-0aaca2bc58eb None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:31.871500 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:31.904598 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd50d22-6429-428d-b8e6-88b2a2334cfd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:31.906100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dd50d22-6429-428d-b8e6-88b2a2334cfd None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:31.957536 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e53c939f-1ca5-4326-ba8d-0aaca2bc58eb None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:31.978085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:32.001790 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8b8a5-1da3-40d3-903e-1ba9e7cb972f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:32.002634 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8b8a5-1da3-40d3-903e-1ba9e7cb972f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:32.002925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8b8a5-1da3-40d3-903e-1ba9e7cb972f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:32.003203 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd8b8a5-1da3-40d3-903e-1ba9e7cb972f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:32.015580 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddd8b8a5-1da3-40d3-903e-1ba9e7cb972f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:32.023430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-ddd8b8a5-1da3-40d3-903e-1ba9e7cb972f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:53:32.023430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:32.023430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:32.023430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:32.023430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:32.023430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:32.023430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:32.023430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:32.023430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:32.023430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:32.023430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:32.023430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:32.023430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:32.023430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:32.023430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:32.023430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:32.024313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:32.024313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:32.024313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:32.024313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:32.024313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:32.024313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:32.024313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:32.024313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:32.024313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:32.024313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:32.024313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:32.024313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:32.024313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:32.024313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:32.024313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:32.024313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:32.025098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:32.025098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:32.025098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:32.025098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:32.025098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:32.025098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:32.025098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:32.025098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:32.025098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:32.025098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:32.025098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:53:32.025098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:32.025098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 230/446] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:53:31 2022] GET /identity/v3/roles/admin => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:53:32.030488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8b8a5-1da3-40d3-903e-1ba9e7cb972f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:32.035114 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 217/447] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 381 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 13:53:32.043875 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd50d22-6429-428d-b8e6-88b2a2334cfd None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:32.118706 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:32.136880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd8b8a5-1da3-40d3-903e-1ba9e7cb972f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:32.150391 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dd50d22-6429-428d-b8e6-88b2a2334cfd None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:32.171382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0f70c-9ed5-46be-b844-e6baff01bb6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:32.172481 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0f70c-9ed5-46be-b844-e6baff01bb6a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:32.172771 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0f70c-9ed5-46be-b844-e6baff01bb6a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:32.173054 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb0f70c-9ed5-46be-b844-e6baff01bb6a None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:32.173815 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0f70c-9ed5-46be-b844-e6baff01bb6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:32.174127 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0f70c-9ed5-46be-b844-e6baff01bb6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:32.175543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb0f70c-9ed5-46be-b844-e6baff01bb6a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:32.184164 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 231/448] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:32.185194 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c0468-d1de-4e44-9396-0ca4c6881419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:32.186054 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c0468-d1de-4e44-9396-0ca4c6881419 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:32.186365 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c0468-d1de-4e44-9396-0ca4c6881419 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:32.186645 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9c0468-d1de-4e44-9396-0ca4c6881419 None admin] PATH_INFO: `/v3/projects/34304bd3ae654c1c976f55ca788ae592` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:32.187881 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c0468-d1de-4e44-9396-0ca4c6881419 None admin] RBAC: Authorizing `identity:get_project(project_id=34304bd3ae654c1c976f55ca788ae592)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:32.189600 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9c0468-d1de-4e44-9396-0ca4c6881419 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:32.189720 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0f70c-9ed5-46be-b844-e6baff01bb6a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:32.195515 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 218/449] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:32 2022] GET /identity/v3/projects/34304bd3ae654c1c976f55ca788ae592 => generated 302 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:32.196463 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c0468-d1de-4e44-9396-0ca4c6881419 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:32.228943 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:32.289518 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb0f70c-9ed5-46be-b844-e6baff01bb6a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:32.296049 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9c0468-d1de-4e44-9396-0ca4c6881419 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:32.323696 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018c4b0-ace1-49e7-8622-a2a65e7bfb5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:32.324824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018c4b0-ace1-49e7-8622-a2a65e7bfb5c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:32.324824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018c4b0-ace1-49e7-8622-a2a65e7bfb5c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:32.325010 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2018c4b0-ace1-49e7-8622-a2a65e7bfb5c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:32.330076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376192f3-43a1-4c75-bd13-1958f0c6e9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:32.330961 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376192f3-43a1-4c75-bd13-1958f0c6e9ae None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:32.331238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376192f3-43a1-4c75-bd13-1958f0c6e9ae None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:32.331512 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376192f3-43a1-4c75-bd13-1958f0c6e9ae None admin] PATH_INFO: `/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:32.332683 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376192f3-43a1-4c75-bd13-1958f0c6e9ae None admin] RBAC: Authorizing `identity:get_group(group_id=1f6dd111ebd740e1a7ae97ca6211ffd0)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:32.334116 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376192f3-43a1-4c75-bd13-1958f0c6e9ae None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:32.336081 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 219/450] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 11 13:53:32 2022] GET /identity/v3/groups/1f6dd111ebd740e1a7ae97ca6211ffd0 => generated 226 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:32.341451 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376192f3-43a1-4c75-bd13-1958f0c6e9ae None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:32.352770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:32.393794 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2018c4b0-ace1-49e7-8622-a2a65e7bfb5c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:32.396160 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2018c4b0-ace1-49e7-8622-a2a65e7bfb5c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:32.436598 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:32.444969 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376192f3-43a1-4c75-bd13-1958f0c6e9ae None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:32.479460 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf531b-9a43-47e3-b0a1-ab1ad8d6d758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:32.480116 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf531b-9a43-47e3-b0a1-ab1ad8d6d758 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:32.480403 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf531b-9a43-47e3-b0a1-ab1ad8d6d758 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:32.480683 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5bf531b-9a43-47e3-b0a1-ab1ad8d6d758 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:32.481523 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf531b-9a43-47e3-b0a1-ab1ad8d6d758 None admin] RBAC: Adding query filter params (group.id=1f6dd111ebd740e1a7ae97ca6211ffd0, role.id=37f000302df54045892d35f57104660a, scope.project.id=34304bd3ae654c1c976f55ca788ae592) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:32.481839 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf531b-9a43-47e3-b0a1-ab1ad8d6d758 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:32.483312 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5bf531b-9a43-47e3-b0a1-ab1ad8d6d758 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:32.489650 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 220/451] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 11 13:53:32 2022] GET /identity/v3/role_assignments?group.id=1f6dd111ebd740e1a7ae97ca6211ffd0&scope.project.id=34304bd3ae654c1c976f55ca788ae592&role.id=37f000302df54045892d35f57104660a => generated 611 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:32.499733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 232/452] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 13:53:32.505641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf531b-9a43-47e3-b0a1-ab1ad8d6d758 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:32.581136 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:32.612685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5bf531b-9a43-47e3-b0a1-ab1ad8d6d758 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:32.647437 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791796e-86b5-4ca9-b856-151a910e318e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:32.648310 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791796e-86b5-4ca9-b856-151a910e318e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:32.648600 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791796e-86b5-4ca9-b856-151a910e318e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:32.648885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7791796e-86b5-4ca9-b856-151a910e318e None admin] PATH_INFO: `/v3/projects/5898aef39ca74b7d8b99cb1aa700e9c2` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:32.650092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7791796e-86b5-4ca9-b856-151a910e318e None admin] RBAC: Authorizing `identity:get_project(project_id=5898aef39ca74b7d8b99cb1aa700e9c2)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:32.651590 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7791796e-86b5-4ca9-b856-151a910e318e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:32.653306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 221/453] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:53:32 2022] GET /identity/v3/projects/5898aef39ca74b7d8b99cb1aa700e9c2 => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:32.659327 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018c4b0-ace1-49e7-8622-a2a65e7bfb5c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:32.772973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2018c4b0-ace1-49e7-8622-a2a65e7bfb5c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:32.809244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f51ccdf-6f4a-4806-9ae5-60cbc669ae18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:32.810108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51ccdf-6f4a-4806-9ae5-60cbc669ae18 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:32.810449 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51ccdf-6f4a-4806-9ae5-60cbc669ae18 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:32.810733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f51ccdf-6f4a-4806-9ae5-60cbc669ae18 None admin] PATH_INFO: `/v3/groups/701f37c2b4504b3abe423ce570d8f48e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:32.811959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51ccdf-6f4a-4806-9ae5-60cbc669ae18 None admin] RBAC: Authorizing `identity:get_group(group_id=701f37c2b4504b3abe423ce570d8f48e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:32.813399 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f51ccdf-6f4a-4806-9ae5-60cbc669ae18 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:32.815057 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 233/454] 10.222.0.49 () {56 vars in 1180 bytes} [Mon Jul 11 13:53:32 2022] GET /identity/v3/groups/701f37c2b4504b3abe423ce570d8f48e => generated 229 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:32.820905 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791796e-86b5-4ca9-b856-151a910e318e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:32.918998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7791796e-86b5-4ca9-b856-151a910e318e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:32.951191 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4108b8-ddee-43d0-8f30-d2d04bdd576b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:32.951839 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4108b8-ddee-43d0-8f30-d2d04bdd576b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:32.952121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4108b8-ddee-43d0-8f30-d2d04bdd576b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:32.952400 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4108b8-ddee-43d0-8f30-d2d04bdd576b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:32.953812 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4108b8-ddee-43d0-8f30-d2d04bdd576b None admin] RBAC: Adding query filter params (group.id=701f37c2b4504b3abe423ce570d8f48e, role.id=6964639a7f6c41e8a6701ef533151c41, scope.project.id=5898aef39ca74b7d8b99cb1aa700e9c2) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:32.954121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4108b8-ddee-43d0-8f30-d2d04bdd576b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:32.955574 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4108b8-ddee-43d0-8f30-d2d04bdd576b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:32.961354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 222/455] 10.222.0.49 () {56 vars in 1376 bytes} [Mon Jul 11 13:53:32 2022] GET /identity/v3/role_assignments?group.id=701f37c2b4504b3abe423ce570d8f48e&scope.project.id=5898aef39ca74b7d8b99cb1aa700e9c2&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:35.371861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a35e59-cef6-441a-9d21-32eb83143e0d None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:35.372809 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a35e59-cef6-441a-9d21-32eb83143e0d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:35.373313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a35e59-cef6-441a-9d21-32eb83143e0d None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:35.375220 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 234/456] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:35.380745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95384908-49be-4325-be56-091fdf6f247a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:35.381587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95384908-49be-4325-be56-091fdf6f247a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:35.382069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95384908-49be-4325-be56-091fdf6f247a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:35.393083 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:35.407272 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-95384908-49be-4325-be56-091fdf6f247a None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:35.408459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-95384908-49be-4325-be56-091fdf6f247a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:35.491009 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 223/457] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:35.522803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a35e59-cef6-441a-9d21-32eb83143e0d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:35.533087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:35.651008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a35e59-cef6-441a-9d21-32eb83143e0d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:35.693525 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b173ed-c106-4b34-96c6-7d906ea7f73a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:35.694146 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba919c-9fb1-40cd-9083-e6670efd6497 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:35.694339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b173ed-c106-4b34-96c6-7d906ea7f73a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:35.694629 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b173ed-c106-4b34-96c6-7d906ea7f73a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:35.694903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b173ed-c106-4b34-96c6-7d906ea7f73a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:35.695097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba919c-9fb1-40cd-9083-e6670efd6497 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:35.695515 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ba919c-9fb1-40cd-9083-e6670efd6497 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:35.697323 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 224/458] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:35.703400 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5eab2-89bd-47ec-b1eb-5aac329e26e5 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:35.703874 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5eab2-89bd-47ec-b1eb-5aac329e26e5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:35.704293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f5eab2-89bd-47ec-b1eb-5aac329e26e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:35.710436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b173ed-c106-4b34-96c6-7d906ea7f73a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:35.710436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2b173ed-c106-4b34-96c6-7d906ea7f73a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:35.730547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-82f5eab2-89bd-47ec-b1eb-5aac329e26e5 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:35.732070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-82f5eab2-89bd-47ec-b1eb-5aac329e26e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:35.733319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-d2b173ed-c106-4b34-96c6-7d906ea7f73a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:35.733319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:35.733319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:35.733319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:35.733319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:35.733319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:35.733319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:35.733319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:35.733319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:35.733319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:35.733319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:35.733319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:35.733319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:35.733319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:35.733319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:35.734228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:35.734228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:35.734228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:35.734228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:35.734228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:35.734228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:35.734228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:35.734228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:35.734228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:35.734228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:35.734228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:35.734228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:35.734228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:35.734228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:35.734228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:35.734228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:35.734228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:35.735602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:35.735602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:35.735602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:35.735602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:35.735602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 235/459] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:35.737956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac6876-8b8d-4fe7-89fe-9caf958bc9e0 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:35.737956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac6876-8b8d-4fe7-89fe-9caf958bc9e0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:35.737956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fac6876-8b8d-4fe7-89fe-9caf958bc9e0 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:35.738602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 236/460] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:35.745319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac6876-8b8d-4fe7-89fe-9caf958bc9e0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:35.814575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:35.822393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 225/461] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:53:35.824003 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e0f41-fbee-4784-95d7-a340bf94cbc7 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:35.824541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e0f41-fbee-4784-95d7-a340bf94cbc7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:35.825020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e0f41-fbee-4784-95d7-a340bf94cbc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:35.844592 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-499e0f41-fbee-4784-95d7-a340bf94cbc7 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:35.846126 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-499e0f41-fbee-4784-95d7-a340bf94cbc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:35.884363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fac6876-8b8d-4fe7-89fe-9caf958bc9e0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:35.917860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:35.921875 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 226/462] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:35.922209 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166302b-cc1a-4149-993c-cd0eb51e4b42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:35.926481 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166302b-cc1a-4149-993c-cd0eb51e4b42 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:35.926481 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166302b-cc1a-4149-993c-cd0eb51e4b42 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:35.926481 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5166302b-cc1a-4149-993c-cd0eb51e4b42 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:35.926481 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5166302b-cc1a-4149-993c-cd0eb51e4b42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:35.926481 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5166302b-cc1a-4149-993c-cd0eb51e4b42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:35.926481 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5166302b-cc1a-4149-993c-cd0eb51e4b42 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:35.939214 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e0f41-fbee-4784-95d7-a340bf94cbc7 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:35.939498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 237/463] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:35.946233 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166302b-cc1a-4149-993c-cd0eb51e4b42 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:36.069089 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-499e0f41-fbee-4784-95d7-a340bf94cbc7 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:36.072809 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5166302b-cc1a-4149-993c-cd0eb51e4b42 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:36.092790 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:36.110665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151bed4d-96f0-4058-8f98-bb3b9419debd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:36.111473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151bed4d-96f0-4058-8f98-bb3b9419debd None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:36.111753 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151bed4d-96f0-4058-8f98-bb3b9419debd None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:36.112031 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151bed4d-96f0-4058-8f98-bb3b9419debd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:36.115714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682aa99-8265-41a2-8c04-22e94bb0f06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:36.116587 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682aa99-8265-41a2-8c04-22e94bb0f06e None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:36.117022 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682aa99-8265-41a2-8c04-22e94bb0f06e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:36.117448 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8682aa99-8265-41a2-8c04-22e94bb0f06e None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:36.118332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682aa99-8265-41a2-8c04-22e94bb0f06e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:36.120078 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8682aa99-8265-41a2-8c04-22e94bb0f06e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:36.122952 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151bed4d-96f0-4058-8f98-bb3b9419debd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:36.124942 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151bed4d-96f0-4058-8f98-bb3b9419debd None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:36.145357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-151bed4d-96f0-4058-8f98-bb3b9419debd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:36.145357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:36.145357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:36.145357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:36.145357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:36.145357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:36.145357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:36.145357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:36.145357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:36.145357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:36.145357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:36.145357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:36.145357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:36.145357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:36.145357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:36.146549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:36.146549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:36.146549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:36.146549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:36.146549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:36.146549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:36.146549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:36.146549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:36.146549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:36.146549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:36.146549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:36.146549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:36.146549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:36.146549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:36.146549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:36.146549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:36.146549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:36.147643 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:36.147643 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:36.147643 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:36.147643 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:36.147643 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 227/464] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 224 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:36.147923 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151bed4d-96f0-4058-8f98-bb3b9419debd None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:36.154787 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 238/465] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 11 13:53:35 2022] POST /identity/v3/users => generated 246 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 13:53:36.155851 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682aa99-8265-41a2-8c04-22e94bb0f06e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:36.239079 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:36.313953 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151bed4d-96f0-4058-8f98-bb3b9419debd None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:36.327412 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8682aa99-8265-41a2-8c04-22e94bb0f06e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:36.356317 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc39ce3-0601-4ac9-b6ec-15429899b78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:36.356317 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc39ce3-0601-4ac9-b6ec-15429899b78f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:36.356317 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc39ce3-0601-4ac9-b6ec-15429899b78f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:36.356673 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc39ce3-0601-4ac9-b6ec-15429899b78f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:36.370538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc39ce3-0601-4ac9-b6ec-15429899b78f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:36.372033 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc39ce3-0601-4ac9-b6ec-15429899b78f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:36.381061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a823f1-be69-4578-87f7-abab155e1265 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:36.382822 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a823f1-be69-4578-87f7-abab155e1265 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:36.383307 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a823f1-be69-4578-87f7-abab155e1265 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:36.384013 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a823f1-be69-4578-87f7-abab155e1265 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:36.385396 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a823f1-be69-4578-87f7-abab155e1265 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:36.385968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a823f1-be69-4578-87f7-abab155e1265 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:36.388527 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60a823f1-be69-4578-87f7-abab155e1265 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:36.393362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-fcc39ce3-0601-4ac9-b6ec-15429899b78f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:36.393362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:36.393362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:36.393362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:36.393362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:36.393362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:36.393362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:36.393362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:36.393362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:36.393362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:36.393362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:36.393362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:36.393362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:36.393362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:36.393362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:36.394817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:36.394817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:36.394817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:36.394817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:36.394817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:36.394817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:36.394817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:36.394817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:36.394817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:36.394817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:36.394817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:36.394817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:36.394817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:36.394817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:36.394817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:36.394817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:36.394817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:36.395899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:36.395899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:36.395899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:36.395899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:36.395899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 228/466] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 247 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:36.395899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4cd47-e92c-406e-b4a2-25849744750c None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:36.396296 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4cd47-e92c-406e-b4a2-25849744750c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:36.396296 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ae4cd47-e92c-406e-b4a2-25849744750c None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:36.401835 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 229/467] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:36 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:36.402642 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4cd47-e92c-406e-b4a2-25849744750c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:36.415561 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 239/468] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:36.417914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819ab4a-ae0a-4b8f-870d-db1120417d46 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:36.419647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819ab4a-ae0a-4b8f-870d-db1120417d46 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:36.420188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2819ab4a-ae0a-4b8f-870d-db1120417d46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:36.457719 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2819ab4a-ae0a-4b8f-870d-db1120417d46 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:36.459893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2819ab4a-ae0a-4b8f-870d-db1120417d46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:36.504398 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:36.518191 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:36.524121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ae4cd47-e92c-406e-b4a2-25849744750c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:36.566338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f283205-3d93-49e3-983c-16de60965ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:36.567029 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f283205-3d93-49e3-983c-16de60965ec2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:36.567309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f283205-3d93-49e3-983c-16de60965ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:36.567700 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f283205-3d93-49e3-983c-16de60965ec2 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:36.568110 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 240/469] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:36.569307 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819ab4a-ae0a-4b8f-870d-db1120417d46 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:36.572954 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f283205-3d93-49e3-983c-16de60965ec2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:36.573345 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f283205-3d93-49e3-983c-16de60965ec2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:36.575242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f283205-3d93-49e3-983c-16de60965ec2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:36.609758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 230/470] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:36.616577 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f283205-3d93-49e3-983c-16de60965ec2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:36.699339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2819ab4a-ae0a-4b8f-870d-db1120417d46 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:36.743603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6253b-07f3-47d2-ba9e-9579ce327e00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:36.744524 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6253b-07f3-47d2-ba9e-9579ce327e00 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:36.744652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6253b-07f3-47d2-ba9e-9579ce327e00 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:36.744874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab6253b-07f3-47d2-ba9e-9579ce327e00 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:36.745599 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6253b-07f3-47d2-ba9e-9579ce327e00 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:36.747107 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ab6253b-07f3-47d2-ba9e-9579ce327e00 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:36.773142 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 241/471] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 11 13:53:36 2022] POST /identity/v3/users => generated 248 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:53:36.774423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6253b-07f3-47d2-ba9e-9579ce327e00 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:36.777682 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f283205-3d93-49e3-983c-16de60965ec2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:36.801441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:36.820964 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32de8a0-6d74-4107-98ae-51a4cd54af10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:36.822008 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32de8a0-6d74-4107-98ae-51a4cd54af10 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:36.822529 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32de8a0-6d74-4107-98ae-51a4cd54af10 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:36.823013 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b32de8a0-6d74-4107-98ae-51a4cd54af10 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:36.824020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32de8a0-6d74-4107-98ae-51a4cd54af10 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:36.825772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b32de8a0-6d74-4107-98ae-51a4cd54af10 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:36.870135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 231/472] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 11 13:53:36 2022] POST /identity/v3/users => generated 248 bytes in 254 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:53:36.904556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab6253b-07f3-47d2-ba9e-9579ce327e00 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:36.947065 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d074793-e274-4faa-9e59-b0a23a78cb8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:36.947906 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d074793-e274-4faa-9e59-b0a23a78cb8e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:36.948195 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d074793-e274-4faa-9e59-b0a23a78cb8e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:36.948465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d074793-e274-4faa-9e59-b0a23a78cb8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:36.961658 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d074793-e274-4faa-9e59-b0a23a78cb8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:36.963167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d074793-e274-4faa-9e59-b0a23a78cb8e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:36.983009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-1d074793-e274-4faa-9e59-b0a23a78cb8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:36.983009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:36.983009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:36.983009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:36.983009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:36.983009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:36.983009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:36.983009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:36.983009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:36.983009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:36.983009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:36.983009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:36.983009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:36.983009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:36.983009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:36.983974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:36.983974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:36.983974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:36.983974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:36.983974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:36.983974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:36.983974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:36.983974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:36.983974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:36.983974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:36.983974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:36.983974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:36.983974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:36.983974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:36.983974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:36.983974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:36.983974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:36.984813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:36.984813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:36.984813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:36.984813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:36.989261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 242/473] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:36.991103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32de8a0-6d74-4107-98ae-51a4cd54af10 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:37.010766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:37.116573 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b32de8a0-6d74-4107-98ae-51a4cd54af10 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:37.149538 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21b14d-0410-4948-bf4a-9bd4ccb6e6ed None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:37.149826 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21b14d-0410-4948-bf4a-9bd4ccb6e6ed None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:37.150105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc21b14d-0410-4948-bf4a-9bd4ccb6e6ed None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:37.152046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 243/474] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:37.157690 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eda4f1-fe08-417e-b69a-12b251190cfe None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:37.158219 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eda4f1-fe08-417e-b69a-12b251190cfe None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:37.158708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89eda4f1-fe08-417e-b69a-12b251190cfe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:37.161740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98141722-6e26-47a2-bbd1-42160846f425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:37.161740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98141722-6e26-47a2-bbd1-42160846f425 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:37.161740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98141722-6e26-47a2-bbd1-42160846f425 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:37.162030 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98141722-6e26-47a2-bbd1-42160846f425 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:37.163201 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98141722-6e26-47a2-bbd1-42160846f425 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:37.163201 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98141722-6e26-47a2-bbd1-42160846f425 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:37.164593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98141722-6e26-47a2-bbd1-42160846f425 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:37.182350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 232/475] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:37.184873 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98141722-6e26-47a2-bbd1-42160846f425 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:37.185956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-89eda4f1-fe08-417e-b69a-12b251190cfe None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:37.187809 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-89eda4f1-fe08-417e-b69a-12b251190cfe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:37.274916 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 244/476] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:53:37.287383 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:37.298374 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eda4f1-fe08-417e-b69a-12b251190cfe None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:37.300746 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:37.338143 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98141722-6e26-47a2-bbd1-42160846f425 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:37.412507 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e46725-ae0e-4497-8416-407d894c8986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:37.413253 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e46725-ae0e-4497-8416-407d894c8986 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:37.413549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e46725-ae0e-4497-8416-407d894c8986 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:37.413831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43e46725-ae0e-4497-8416-407d894c8986 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:37.414619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e46725-ae0e-4497-8416-407d894c8986 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:37.417379 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43e46725-ae0e-4497-8416-407d894c8986 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:37.455774 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 233/477] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 11 13:53:37 2022] POST /identity/v3/users => generated 249 bytes in 271 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 13:53:37.503454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89eda4f1-fe08-417e-b69a-12b251190cfe None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:37.556471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020280e3-14f8-4a2c-827c-08f2b3417217 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:37.557178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020280e3-14f8-4a2c-827c-08f2b3417217 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:37.557470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020280e3-14f8-4a2c-827c-08f2b3417217 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:37.557761 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020280e3-14f8-4a2c-827c-08f2b3417217 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:37.558643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020280e3-14f8-4a2c-827c-08f2b3417217 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:37.560297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020280e3-14f8-4a2c-827c-08f2b3417217 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:37.575787 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.sql.core [None req-020280e3-14f8-4a2c-827c-08f2b3417217 None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 11 13:53:37.575787 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 11 13:53:37.575787 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [parameters: {'id': 'b5ebecaa980545b2884608b03b5160f5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 11 13:53:37.575787 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80248) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 11 13:53:37.598228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-020280e3-14f8-4a2c-827c-08f2b3417217 None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 11 13:53:37.598228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:37.598228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 13:53:37.598228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 13:53:37.598228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 13:53:37.598228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 13:53:37.598228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 13:53:37.598228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 13:53:37.598228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 13:53:37.598228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application conn.query(q) Jul 11 13:53:37.598228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 13:53:37.598228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 13:53:37.598228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 13:53:37.598228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application result.read() Jul 11 13:53:37.598228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 13:53:37.599154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 13:53:37.599154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 13:53:37.599154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 13:53:37.599154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 13:53:37.599154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 13:53:37.599154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 13:53:37.599154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 13:53:37.599154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 11 13:53:37.599154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:37.599154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 11 13:53:37.599154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:37.599154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:37.599154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 11 13:53:37.599154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 11 13:53:37.599154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 11 13:53:37.599154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return ref.to_dict() Jul 11 13:53:37.599154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 13:53:37.599154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application next(self.gen) Jul 11 13:53:37.600138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 11 13:53:37.600138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application yield resource Jul 11 13:53:37.600138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 13:53:37.600138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application next(self.gen) Jul 11 13:53:37.600138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 11 13:53:37.600138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.session.rollback() Jul 11 13:53:37.600138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 11 13:53:37.600138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.force_reraise() Jul 11 13:53:37.600138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 11 13:53:37.600138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise self.value Jul 11 13:53:37.600138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 11 13:53:37.600138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 11 13:53:37.600138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 11 13:53:37.600138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application session.commit() Jul 11 13:53:37.600138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 11 13:53:37.600138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 11 13:53:37.600138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 11 13:53:37.600988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._prepare_impl() Jul 11 13:53:37.600988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 11 13:53:37.600988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.session.flush() Jul 11 13:53:37.600988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 11 13:53:37.600988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._flush(objects) Jul 11 13:53:37.600988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 11 13:53:37.600988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 11 13:53:37.600988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 11 13:53:37.600988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application compat.raise_( Jul 11 13:53:37.600988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 13:53:37.600988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception Jul 11 13:53:37.600988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 11 13:53:37.600988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application flush_context.execute() Jul 11 13:53:37.600988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 11 13:53:37.600988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rec.execute(self) Jul 11 13:53:37.600988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 11 13:53:37.600988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 11 13:53:37.601835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 11 13:53:37.601835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 11 13:53:37.601835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 11 13:53:37.601835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 11 13:53:37.601835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 11 13:53:37.601835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 11 13:53:37.601835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 11 13:53:37.601835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 11 13:53:37.601835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 11 13:53:37.601835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 11 13:53:37.601835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 11 13:53:37.601835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 11 13:53:37.601835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 11 13:53:37.601835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 11 13:53:37.601835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 13:53:37.601835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception Jul 11 13:53:37.602685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 13:53:37.602685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 13:53:37.602685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 13:53:37.602685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 13:53:37.602685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 13:53:37.602685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 13:53:37.602685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 13:53:37.602685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application conn.query(q) Jul 11 13:53:37.602685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 13:53:37.602685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 13:53:37.602685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 13:53:37.602685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application result.read() Jul 11 13:53:37.602685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 13:53:37.602685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 13:53:37.602685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 13:53:37.602685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 13:53:37.603450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 13:53:37.603450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 13:53:37.603450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 13:53:37.603450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 13:53:37.603450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 11 13:53:37.603450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 11 13:53:37.603450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [parameters: {'id': 'b5ebecaa980545b2884608b03b5160f5', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 11 13:53:37.603450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 11 13:53:37.603450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:37.603450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:53:37.603450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:37.603450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:37.603450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:37.603450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:37.603450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:37.603450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:37.604222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:37.604222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:37.604222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:37.604222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:37.604222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:37.604222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:37.604222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 11 13:53:37.604222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 11 13:53:37.604222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:37.604222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:37.604222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 11 13:53:37.604222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 11 13:53:37.604222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 11 13:53:37.604222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 11 13:53:37.604222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 11 13:53:37.604222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:37.605072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 245/478] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Jul 11 13:53:37 2022] POST /identity/v3/roles => generated 144 bytes in 302 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:53:37.613231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e46725-ae0e-4497-8416-407d894c8986 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:37.724128 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43e46725-ae0e-4497-8416-407d894c8986 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:37.757029 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:37.769070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9df0ebb2-4a4d-4d89-9a13-91a64df1e514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:37.769905 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0ebb2-4a4d-4d89-9a13-91a64df1e514 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:37.770191 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0ebb2-4a4d-4d89-9a13-91a64df1e514 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:37.770493 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df0ebb2-4a4d-4d89-9a13-91a64df1e514 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:37.785689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9df0ebb2-4a4d-4d89-9a13-91a64df1e514 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:37.793911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-9df0ebb2-4a4d-4d89-9a13-91a64df1e514 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:53:37.793911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:37.793911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:37.793911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:37.793911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:37.793911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:37.793911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:37.793911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:37.793911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:37.793911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:37.793911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:37.793911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:37.793911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:37.793911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:37.793911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:37.793911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:37.795973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:37.795973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:37.795973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:37.795973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:37.795973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:37.795973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:37.795973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:37.795973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:37.795973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:37.795973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:37.795973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:37.795973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:37.795973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:37.795973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:37.795973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:37.795973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:37.798584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:37.798584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:37.798584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:37.798584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:37.798584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:37.798584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:37.798584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:37.798584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:37.798584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:37.798584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:37.798584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 11 13:53:37.798584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:37.798584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 234/479] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:53:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:37.802850 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020280e3-14f8-4a2c-827c-08f2b3417217 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:37.893948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:37.939228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020280e3-14f8-4a2c-827c-08f2b3417217 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:37.992914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea70b2f-92eb-480b-ba85-946974e93d52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:37.994072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea70b2f-92eb-480b-ba85-946974e93d52 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:37.994599 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea70b2f-92eb-480b-ba85-946974e93d52 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:37.995105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea70b2f-92eb-480b-ba85-946974e93d52 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:37.996549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea70b2f-92eb-480b-ba85-946974e93d52 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:37.997125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea70b2f-92eb-480b-ba85-946974e93d52 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:37.999834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ea70b2f-92eb-480b-ba85-946974e93d52 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:38.015987 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 246/480] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:38.698127 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7af56d-df08-4eb1-b91d-f8f636175413 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:38.698652 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7af56d-df08-4eb1-b91d-f8f636175413 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:38.698726 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7af56d-df08-4eb1-b91d-f8f636175413 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:38.700102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 235/481] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:38.714771 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2206347-ed0b-49e6-98bb-e0c89e9475ab None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:38.715181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2206347-ed0b-49e6-98bb-e0c89e9475ab None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:38.715369 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2206347-ed0b-49e6-98bb-e0c89e9475ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:38.739227 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b2206347-ed0b-49e6-98bb-e0c89e9475ab None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:38.740431 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b2206347-ed0b-49e6-98bb-e0c89e9475ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:38.840667 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 247/482] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:38.887876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7af56d-df08-4eb1-b91d-f8f636175413 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:39.019176 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7af56d-df08-4eb1-b91d-f8f636175413 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:39.063684 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622231d0-6cc9-45f1-88a3-3fad986b742f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:39.064504 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622231d0-6cc9-45f1-88a3-3fad986b742f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:39.064789 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622231d0-6cc9-45f1-88a3-3fad986b742f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:39.065065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622231d0-6cc9-45f1-88a3-3fad986b742f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:39.084216 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-622231d0-6cc9-45f1-88a3-3fad986b742f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:39.092524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-622231d0-6cc9-45f1-88a3-3fad986b742f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:39.092524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:39.092524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:39.092524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:39.092524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:39.092524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:39.092524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:39.092524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:39.092524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:39.092524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:39.092524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:39.092524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:39.092524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:39.092524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:39.092524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:39.092524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:39.094002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:39.094002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:39.094002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:39.094002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:39.094002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:39.094002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:39.094002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:39.094002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:39.094002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:39.094002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:39.094002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:39.094002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:39.094002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:39.094002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:39.094002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:39.094002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:39.094833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:39.094833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:39.094833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:39.094833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:39.094833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:39.094833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:39.094833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:39.094833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:39.094833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:39.094833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:39.094833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:39.094833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:39.094833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 236/483] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:53:38 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:39.100425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2206347-ed0b-49e6-98bb-e0c89e9475ab None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:39.193058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:39.246346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2206347-ed0b-49e6-98bb-e0c89e9475ab None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:39.289666 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73260387-7df1-41d4-a2e8-2fdb79a357fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:39.290282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73260387-7df1-41d4-a2e8-2fdb79a357fa None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:39.290592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73260387-7df1-41d4-a2e8-2fdb79a357fa None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:39.290872 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73260387-7df1-41d4-a2e8-2fdb79a357fa None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:39.291653 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73260387-7df1-41d4-a2e8-2fdb79a357fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:39.291964 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73260387-7df1-41d4-a2e8-2fdb79a357fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:39.293381 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73260387-7df1-41d4-a2e8-2fdb79a357fa None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:39.304246 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 248/484] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:53:39 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:39.309928 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622231d0-6cc9-45f1-88a3-3fad986b742f None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:39.378226 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:39.444996 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622231d0-6cc9-45f1-88a3-3fad986b742f None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:39.489664 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c6748f7-b18c-4b9a-9090-56d00165ac7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:39.490476 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6748f7-b18c-4b9a-9090-56d00165ac7a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:39.490762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6748f7-b18c-4b9a-9090-56d00165ac7a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:39.491050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c6748f7-b18c-4b9a-9090-56d00165ac7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:39.503399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6748f7-b18c-4b9a-9090-56d00165ac7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:39.504890 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c6748f7-b18c-4b9a-9090-56d00165ac7a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:39.534124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-7c6748f7-b18c-4b9a-9090-56d00165ac7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:39.534124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:39.534124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:39.534124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:39.534124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:39.534124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:39.534124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:39.534124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:39.534124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:39.534124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:39.534124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:39.534124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:39.534124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:39.534124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:39.534124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:39.535101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:39.535101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:39.535101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:39.535101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:39.535101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:39.535101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:39.535101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:39.535101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:39.535101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:39.535101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:39.535101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:39.535101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:39.535101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:39.535101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:39.535101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:39.535101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:39.535101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:39.535923 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:39.535923 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:39.535923 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:39.535923 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:39.535923 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 237/485] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:39.541445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73260387-7df1-41d4-a2e8-2fdb79a357fa None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:39.613407 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26c6c5-515e-48bb-98c6-aac53fcc269b None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:39.613713 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26c6c5-515e-48bb-98c6-aac53fcc269b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:39.613967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f26c6c5-515e-48bb-98c6-aac53fcc269b None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:39.615664 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 238/486] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:39.621320 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fb305-2361-438c-ac6c-dcc3de6848d3 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:39.621616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fb305-2361-438c-ac6c-dcc3de6848d3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:39.621888 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8fb305-2361-438c-ac6c-dcc3de6848d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:39.645365 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-be8fb305-2361-438c-ac6c-dcc3de6848d3 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:39.647032 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-be8fb305-2361-438c-ac6c-dcc3de6848d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:39.695689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73260387-7df1-41d4-a2e8-2fdb79a357fa None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:39.717992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:39.738217 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f33eaf-ac3a-42cf-9c9c-2895b94b9656 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:39.738504 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 239/487] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:53:39.740610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46e4a7-9abe-4dcb-b950-68c7ff16f0b4 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:39.741154 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46e4a7-9abe-4dcb-b950-68c7ff16f0b4 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:39.741706 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d46e4a7-9abe-4dcb-b950-68c7ff16f0b4 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:39.747256 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f33eaf-ac3a-42cf-9c9c-2895b94b9656 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:39.747961 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 240/488] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:39.748104 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f33eaf-ac3a-42cf-9c9c-2895b94b9656 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:39.748300 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f33eaf-ac3a-42cf-9c9c-2895b94b9656 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:39.749733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f33eaf-ac3a-42cf-9c9c-2895b94b9656 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:39.750281 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f33eaf-ac3a-42cf-9c9c-2895b94b9656 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:39.750802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746291a-41e3-49c5-95d5-546e3ce37e3d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:39.751342 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746291a-41e3-49c5-95d5-546e3ce37e3d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:39.751852 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9746291a-41e3-49c5-95d5-546e3ce37e3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:39.751973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f33eaf-ac3a-42cf-9c9c-2895b94b9656 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:39.780673 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 249/489] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:39.781821 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f33eaf-ac3a-42cf-9c9c-2895b94b9656 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:39.781933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9746291a-41e3-49c5-95d5-546e3ce37e3d None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:39.783497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9746291a-41e3-49c5-95d5-546e3ce37e3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:39.817351 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:39.886748 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 241/490] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:39.888602 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746291a-41e3-49c5-95d5-546e3ce37e3d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:39.901166 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f33eaf-ac3a-42cf-9c9c-2895b94b9656 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:39.942230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ac51d-fa17-4e8e-b1a9-172873f63ef6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:39.943732 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ac51d-fa17-4e8e-b1a9-172873f63ef6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:39.944648 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ac51d-fa17-4e8e-b1a9-172873f63ef6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:39.945373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ac51d-fa17-4e8e-b1a9-172873f63ef6 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:39.959953 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ac51d-fa17-4e8e-b1a9-172873f63ef6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:39.970814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-780ac51d-fa17-4e8e-b1a9-172873f63ef6 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:39.970814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:39.970814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:39.970814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:39.970814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:39.970814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:39.970814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:39.970814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:39.970814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:39.970814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:39.970814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:39.970814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:39.970814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:39.970814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:39.970814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:39.970814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:39.971756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:39.971756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:39.971756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:39.971756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:39.971756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:39.971756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:39.971756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:39.971756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:39.971756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:39.971756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:39.971756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:39.971756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:39.971756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:39.971756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:39.971756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:39.971756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:39.972540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:39.972540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:39.972540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:39.972540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:39.972540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:39.972540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:39.972540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:39.972540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:39.972540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:39.972540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:39.972540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:39.972540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:39.975344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 250/491] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:53:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:39.976461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ac51d-fa17-4e8e-b1a9-172873f63ef6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:40.033097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:40.036013 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9746291a-41e3-49c5-95d5-546e3ce37e3d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:40.081282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1836d8-2af1-44fb-a371-e656ed075f9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:40.082100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1836d8-2af1-44fb-a371-e656ed075f9c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:40.082415 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1836d8-2af1-44fb-a371-e656ed075f9c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:40.082694 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e1836d8-2af1-44fb-a371-e656ed075f9c None admin] PATH_INFO: `/v3/users/nova` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:40.092566 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1836d8-2af1-44fb-a371-e656ed075f9c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:40.094346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e1836d8-2af1-44fb-a371-e656ed075f9c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:40.116593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-0e1836d8-2af1-44fb-a371-e656ed075f9c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 13:53:40.116593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:40.116593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:40.116593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:40.116593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:40.116593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:40.116593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:40.116593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:40.116593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:40.116593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:40.116593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:40.116593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:40.116593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:53:40.116593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:53:40.116593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:53:40.116593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:53:40.118350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:40.118350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:40.118350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:53:40.118350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:40.118350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:53:40.118350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:40.118350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:40.118350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:40.118350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:40.118350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:40.118350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:40.118350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:40.118350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:40.118350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:40.118350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:40.118350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:40.118350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:40.118350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:40.120133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:40.120133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:40.120133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:53:40.120133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:53:40.120133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:53:40.120133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:53:40.120133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:53:40.120133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:53:40.120133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 13:53:40.120133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:40.120133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 242/492] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Jul 11 13:53:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:40.120133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1836d8-2af1-44fb-a371-e656ed075f9c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:40.131165 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ac51d-fa17-4e8e-b1a9-172873f63ef6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:40.154086 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:40.185990 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122fccc-bc57-4b73-8dc1-5082eccd4324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:40.187458 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122fccc-bc57-4b73-8dc1-5082eccd4324 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:40.187746 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122fccc-bc57-4b73-8dc1-5082eccd4324 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:40.188024 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6122fccc-bc57-4b73-8dc1-5082eccd4324 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:40.202917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6122fccc-bc57-4b73-8dc1-5082eccd4324 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:40.211257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-6122fccc-bc57-4b73-8dc1-5082eccd4324 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:40.211257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:40.211257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:40.211257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:40.211257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:40.211257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:40.211257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:40.211257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:40.211257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:40.211257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:40.211257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:40.211257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:40.211257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:40.211257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:40.211257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:40.211257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:40.212335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:40.212335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:40.212335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:40.212335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:40.212335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:40.212335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:40.212335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:40.212335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:40.212335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:40.212335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:40.212335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:40.212335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:40.212335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:40.212335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:40.212335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:40.212335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:40.213865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:40.213865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:40.213865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:40.213865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:40.213865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:40.213865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:40.213865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:40.213865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:40.213865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:40.213865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:40.213865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:40.213865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:40.213865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 251/493] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:53:39 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:40.213865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122fccc-bc57-4b73-8dc1-5082eccd4324 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:40.274453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e1836d8-2af1-44fb-a371-e656ed075f9c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:40.326050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-183be2cb-5272-49b8-a959-be10084ca31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:40.327213 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183be2cb-5272-49b8-a959-be10084ca31f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:40.328184 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183be2cb-5272-49b8-a959-be10084ca31f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:40.329500 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-183be2cb-5272-49b8-a959-be10084ca31f None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:40.329648 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183be2cb-5272-49b8-a959-be10084ca31f None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:40.329929 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183be2cb-5272-49b8-a959-be10084ca31f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:40.332144 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-183be2cb-5272-49b8-a959-be10084ca31f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:40.333519 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:40.336895 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6122fccc-bc57-4b73-8dc1-5082eccd4324 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:40.350477 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 243/494] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:53:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:40.350477 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a25422-b89a-42fc-96cf-0c5314fffeef None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:40.350477 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a25422-b89a-42fc-96cf-0c5314fffeef None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:40.350477 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a25422-b89a-42fc-96cf-0c5314fffeef None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:40.350477 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 244/495] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:40.350477 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a25422-b89a-42fc-96cf-0c5314fffeef None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:40.388317 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043bcc76-f780-437d-b3ef-1218e5cc72e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:40.389005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bcc76-f780-437d-b3ef-1218e5cc72e2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:40.389290 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bcc76-f780-437d-b3ef-1218e5cc72e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:40.389642 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043bcc76-f780-437d-b3ef-1218e5cc72e2 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:40.390987 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043bcc76-f780-437d-b3ef-1218e5cc72e2 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:40.391297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043bcc76-f780-437d-b3ef-1218e5cc72e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:40.392747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043bcc76-f780-437d-b3ef-1218e5cc72e2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:40.444199 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 252/496] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Jul 11 13:53:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:40.444199 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e89321-5ec6-477a-bf67-fd5a25154ecd None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:40.444199 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e89321-5ec6-477a-bf67-fd5a25154ecd None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:40.444199 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20e89321-5ec6-477a-bf67-fd5a25154ecd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:40.461965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-20e89321-5ec6-477a-bf67-fd5a25154ecd None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:40.463338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-20e89321-5ec6-477a-bf67-fd5a25154ecd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:40.468901 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:40.490610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a25422-b89a-42fc-96cf-0c5314fffeef None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:40.529010 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b584ca2-4a94-40aa-8005-b2e0d7ff85fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:40.529704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b584ca2-4a94-40aa-8005-b2e0d7ff85fa None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:40.529986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b584ca2-4a94-40aa-8005-b2e0d7ff85fa None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:40.530271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b584ca2-4a94-40aa-8005-b2e0d7ff85fa None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:40.531151 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b584ca2-4a94-40aa-8005-b2e0d7ff85fa None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:40.531466 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b584ca2-4a94-40aa-8005-b2e0d7ff85fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:40.532879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b584ca2-4a94-40aa-8005-b2e0d7ff85fa None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:40.547090 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 245/497] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:53:40 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:40.547090 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b584ca2-4a94-40aa-8005-b2e0d7ff85fa None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:40.594414 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 253/498] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:40.594414 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e89321-5ec6-477a-bf67-fd5a25154ecd None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:40.662995 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b584ca2-4a94-40aa-8005-b2e0d7ff85fa None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:40.696624 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:40.705256 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df72d0-5d43-415c-b618-6c98314b5467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:40.706151 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df72d0-5d43-415c-b618-6c98314b5467 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:40.706472 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df72d0-5d43-415c-b618-6c98314b5467 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:40.706753 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94df72d0-5d43-415c-b618-6c98314b5467 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:40.712362 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20e89321-5ec6-477a-bf67-fd5a25154ecd None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:40.720173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df72d0-5d43-415c-b618-6c98314b5467 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:40.721799 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94df72d0-5d43-415c-b618-6c98314b5467 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:40.729162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:40.741495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-94df72d0-5d43-415c-b618-6c98314b5467 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:40.741495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:40.741495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:40.741495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:40.741495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:40.741495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:40.741495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:40.741495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:40.741495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:40.741495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:40.741495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:40.741495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:40.741495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:40.741495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:40.741495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:40.742939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:40.742939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:40.742939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:40.742939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:40.742939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:40.742939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:40.742939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:40.742939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:40.742939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:40.742939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:40.742939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:40.742939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:40.742939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:40.742939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:40.742939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:40.742939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:40.742939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:40.743792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:40.743792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:40.743792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:40.743792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:40.743792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 246/499] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:40.743792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df72d0-5d43-415c-b618-6c98314b5467 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:40.750849 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e18cd5-261e-45fd-8bd2-c4b663df58b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:40.751404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e18cd5-261e-45fd-8bd2-c4b663df58b9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:40.751692 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e18cd5-261e-45fd-8bd2-c4b663df58b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:40.751971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e18cd5-261e-45fd-8bd2-c4b663df58b9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:40.824906 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e18cd5-261e-45fd-8bd2-c4b663df58b9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:40.826402 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e18cd5-261e-45fd-8bd2-c4b663df58b9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:40.853338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94df72d0-5d43-415c-b618-6c98314b5467 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:40.891366 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acc215-dd8c-44a2-a7f0-76fa5291fa6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:40.892176 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acc215-dd8c-44a2-a7f0-76fa5291fa6a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:40.892461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acc215-dd8c-44a2-a7f0-76fa5291fa6a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:40.892740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61acc215-dd8c-44a2-a7f0-76fa5291fa6a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:40.903810 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acc215-dd8c-44a2-a7f0-76fa5291fa6a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:40.905292 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61acc215-dd8c-44a2-a7f0-76fa5291fa6a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:40.917313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:40.923675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-61acc215-dd8c-44a2-a7f0-76fa5291fa6a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:40.923675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:40.923675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:40.923675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:40.923675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:40.923675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:40.923675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:40.923675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:40.923675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:40.923675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:40.923675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:40.923675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:40.923675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:40.923675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:40.923675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:40.924641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:40.924641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:40.924641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:40.924641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:40.924641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:40.924641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:40.924641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:40.924641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:40.924641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:40.924641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:40.924641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:40.924641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:40.924641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:40.924641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:40.924641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:40.924641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:40.924641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:40.925678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:40.925678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:40.925678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:40.925678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:40.925678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 247/500] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:40.925976 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acc215-dd8c-44a2-a7f0-76fa5291fa6a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:40.938629 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 254/501] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:40.941029 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb644a-f231-4493-bd85-218e6d20295b None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:40.941332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb644a-f231-4493-bd85-218e6d20295b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:40.941776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afb644a-f231-4493-bd85-218e6d20295b None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:40.943171 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 255/502] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:40.943884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb644a-f231-4493-bd85-218e6d20295b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:40.983336 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:41.040694 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61acc215-dd8c-44a2-a7f0-76fa5291fa6a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:41.053750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afb644a-f231-4493-bd85-218e6d20295b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:41.079678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8570ba-085a-47e3-990e-f25cec9b1f5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:41.080431 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8570ba-085a-47e3-990e-f25cec9b1f5e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:41.080725 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8570ba-085a-47e3-990e-f25cec9b1f5e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:41.081001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8570ba-085a-47e3-990e-f25cec9b1f5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:41.089579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff41ef-18c2-4cbd-890d-fd283a0ba9d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:41.090344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff41ef-18c2-4cbd-890d-fd283a0ba9d3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:41.090555 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff41ef-18c2-4cbd-890d-fd283a0ba9d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:41.090844 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74ff41ef-18c2-4cbd-890d-fd283a0ba9d3 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:41.091571 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff41ef-18c2-4cbd-890d-fd283a0ba9d3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:41.091699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8570ba-085a-47e3-990e-f25cec9b1f5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:41.091881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff41ef-18c2-4cbd-890d-fd283a0ba9d3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:41.093286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74ff41ef-18c2-4cbd-890d-fd283a0ba9d3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:41.099354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-1b8570ba-085a-47e3-990e-f25cec9b1f5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:41.099354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:41.099354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:41.099354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:41.099354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:41.099354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:41.099354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:41.099354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:41.099354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:41.099354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:41.099354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:41.099354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:41.099354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:41.099354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:41.099354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:41.099354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:41.100513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:41.100513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:41.100513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:41.100513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:41.100513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:41.100513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:41.100513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:41.100513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:41.100513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:41.100513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:41.100513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:41.100513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:41.100513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:41.100513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:41.100513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:41.100513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:41.101429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:41.101429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:41.101429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:41.101429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:41.101429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:41.101429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:41.101429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:41.101429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:41.101429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:41.101429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:41.101429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:41.101429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:41.101429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 248/503] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:53:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:41.101429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8570ba-085a-47e3-990e-f25cec9b1f5e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:41.108996 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 256/504] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:41.109660 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff41ef-18c2-4cbd-890d-fd283a0ba9d3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:41.165373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:41.219782 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8570ba-085a-47e3-990e-f25cec9b1f5e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:41.236181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74ff41ef-18c2-4cbd-890d-fd283a0ba9d3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:41.236831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:41.263786 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edb3589e-ad50-4ee6-b85d-b2a4e65243be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:41.264465 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3589e-ad50-4ee6-b85d-b2a4e65243be None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:41.264747 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3589e-ad50-4ee6-b85d-b2a4e65243be None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:41.265047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edb3589e-ad50-4ee6-b85d-b2a4e65243be None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:41.265820 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3589e-ad50-4ee6-b85d-b2a4e65243be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:41.266123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3589e-ad50-4ee6-b85d-b2a4e65243be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:41.267607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edb3589e-ad50-4ee6-b85d-b2a4e65243be None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:41.281775 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fcc60-1475-4f24-b5d8-9d30cebd419d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:41.282625 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fcc60-1475-4f24-b5d8-9d30cebd419d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:41.282911 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fcc60-1475-4f24-b5d8-9d30cebd419d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:41.283186 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7fcc60-1475-4f24-b5d8-9d30cebd419d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:41.290244 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 249/505] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:41.294047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af929a41-6d38-4567-96be-6219b43d5976 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:41.294548 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af929a41-6d38-4567-96be-6219b43d5976 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:41.295169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af929a41-6d38-4567-96be-6219b43d5976 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:41.296154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7fcc60-1475-4f24-b5d8-9d30cebd419d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:41.298195 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7fcc60-1475-4f24-b5d8-9d30cebd419d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:41.312865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-af929a41-6d38-4567-96be-6219b43d5976 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:41.314076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-af929a41-6d38-4567-96be-6219b43d5976 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:41.318088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-6e7fcc60-1475-4f24-b5d8-9d30cebd419d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:41.318088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:41.318088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:41.318088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:41.318088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:41.318088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:41.318088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:41.318088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:41.318088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:41.318088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:41.318088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:41.318088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:41.318088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:41.318088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:41.318088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:41.319934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:41.319934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:41.319934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:41.319934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:41.319934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:41.319934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:41.319934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:41.319934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:41.319934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:41.319934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:41.319934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:41.319934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:41.319934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:41.319934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:41.319934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:41.319934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:41.319934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:41.321931 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:41.321931 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:41.321931 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:41.321931 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:41.323376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 257/506] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:41.324015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fcc60-1475-4f24-b5d8-9d30cebd419d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:41.396225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 250/507] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:41.396225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af929a41-6d38-4567-96be-6219b43d5976 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:41.443052 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:41.446288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7fcc60-1475-4f24-b5d8-9d30cebd419d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:41.484395 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79595d7e-0c83-4ede-99ff-1270b0ee6bc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:41.485081 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79595d7e-0c83-4ede-99ff-1270b0ee6bc5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:41.485364 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79595d7e-0c83-4ede-99ff-1270b0ee6bc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:41.485643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79595d7e-0c83-4ede-99ff-1270b0ee6bc5 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:41.486450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79595d7e-0c83-4ede-99ff-1270b0ee6bc5 None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:41.486762 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79595d7e-0c83-4ede-99ff-1270b0ee6bc5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:41.488167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79595d7e-0c83-4ede-99ff-1270b0ee6bc5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:41.497250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 258/508] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:53:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:41.498093 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79595d7e-0c83-4ede-99ff-1270b0ee6bc5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:41.525895 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:41.529353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af929a41-6d38-4567-96be-6219b43d5976 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:41.566721 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980aab59-5c5b-4971-ba96-253ee7cddc54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:41.567529 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980aab59-5c5b-4971-ba96-253ee7cddc54 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:41.567808 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980aab59-5c5b-4971-ba96-253ee7cddc54 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:41.568085 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980aab59-5c5b-4971-ba96-253ee7cddc54 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:41.576932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980aab59-5c5b-4971-ba96-253ee7cddc54 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:41.579002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980aab59-5c5b-4971-ba96-253ee7cddc54 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:41.595242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-980aab59-5c5b-4971-ba96-253ee7cddc54 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 13:53:41.595242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:41.595242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:41.595242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:41.595242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:41.595242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:41.595242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:41.595242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:41.595242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:41.595242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:41.595242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:41.595242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:41.595242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:53:41.595242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:53:41.595242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:53:41.595242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:53:41.596920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:41.596920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:41.596920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:53:41.596920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:41.596920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:53:41.596920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:41.596920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:41.596920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:41.596920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:41.596920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:41.596920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:41.596920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:41.596920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:41.596920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:41.596920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:41.596920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:41.596920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:41.596920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:41.599037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:41.599037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:41.599037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:53:41.599037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:53:41.599037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:53:41.599037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:53:41.599037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:53:41.599037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:53:41.599037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 13:53:41.599037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:41.599037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 251/509] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:41 2022] GET /identity/v3/users/cinder => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:41.599037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980aab59-5c5b-4971-ba96-253ee7cddc54 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:41.621928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79595d7e-0c83-4ede-99ff-1270b0ee6bc5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:41.661378 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f4418-604f-4175-bf8f-9fa54671d6d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:41.662098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f4418-604f-4175-bf8f-9fa54671d6d3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:41.662423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f4418-604f-4175-bf8f-9fa54671d6d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:41.662707 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561f4418-604f-4175-bf8f-9fa54671d6d3 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:41.671222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f4418-604f-4175-bf8f-9fa54671d6d3 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:41.672728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561f4418-604f-4175-bf8f-9fa54671d6d3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:41.689935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-561f4418-604f-4175-bf8f-9fa54671d6d3 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 11 13:53:41.689935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:41.689935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:41.689935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:41.689935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:41.689935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:41.689935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:41.689935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:41.689935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:41.689935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:41.689935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:41.689935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:41.689935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:53:41.689935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:53:41.689935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:53:41.689935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:53:41.691061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:41.691061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:41.691061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:53:41.691061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:41.691061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:53:41.691061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:41.691061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:41.691061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:41.691061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:41.691061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:41.691061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:41.691061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:41.691061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:41.691061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:41.691061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:41.691061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:41.691061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:41.691061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:41.692097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:41.692097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:41.692097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:53:41.692097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:53:41.692097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:53:41.692097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:53:41.692097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:53:41.692097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:53:41.692097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 11 13:53:41.692097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:41.692097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 259/510] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:41 2022] GET /identity/v3/users/glance => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:41.692097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f4418-604f-4175-bf8f-9fa54671d6d3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:41.720002 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:41.720245 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980aab59-5c5b-4971-ba96-253ee7cddc54 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:41.759050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1250746-4011-4cbc-a966-32be8f4e280d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:41.759816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1250746-4011-4cbc-a966-32be8f4e280d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:41.759991 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1250746-4011-4cbc-a966-32be8f4e280d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:41.760261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1250746-4011-4cbc-a966-32be8f4e280d None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:41.760997 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1250746-4011-4cbc-a966-32be8f4e280d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:41.761294 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1250746-4011-4cbc-a966-32be8f4e280d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:41.762734 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1250746-4011-4cbc-a966-32be8f4e280d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:41.777819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 252/511] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:41.778724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1250746-4011-4cbc-a966-32be8f4e280d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:41.803585 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:41.807221 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561f4418-604f-4175-bf8f-9fa54671d6d3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:41.849966 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bed9f-b9de-4f46-a9d3-a850605c84ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:41.850819 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bed9f-b9de-4f46-a9d3-a850605c84ac None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:41.851121 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bed9f-b9de-4f46-a9d3-a850605c84ac None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:41.851402 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250bed9f-b9de-4f46-a9d3-a850605c84ac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:41.863037 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bed9f-b9de-4f46-a9d3-a850605c84ac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:41.864495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250bed9f-b9de-4f46-a9d3-a850605c84ac None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:41.882903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-250bed9f-b9de-4f46-a9d3-a850605c84ac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:41.882903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:41.882903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:41.882903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:41.882903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:41.882903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:41.882903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:41.882903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:41.882903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:41.882903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:41.882903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:41.882903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:41.882903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:41.882903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:41.882903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:41.883834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:41.883834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:41.883834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:41.883834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:41.883834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:41.883834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:41.883834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:41.883834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:41.883834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:41.883834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:41.883834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:41.883834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:41.883834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:41.883834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:41.883834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:41.883834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:41.883834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:41.884692 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:41.884692 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:41.884692 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:41.884692 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:41.884692 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 260/512] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:41.885222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bed9f-b9de-4f46-a9d3-a850605c84ac None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:41.908419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1250746-4011-4cbc-a966-32be8f4e280d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:41.946246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e182c-b0b3-439a-9e1b-ae04674e6514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:41.946980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e182c-b0b3-439a-9e1b-ae04674e6514 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:41.947257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e182c-b0b3-439a-9e1b-ae04674e6514 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:41.947538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309e182c-b0b3-439a-9e1b-ae04674e6514 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:41.959215 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e182c-b0b3-439a-9e1b-ae04674e6514 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:41.960727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309e182c-b0b3-439a-9e1b-ae04674e6514 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:41.969568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:41.984369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-309e182c-b0b3-439a-9e1b-ae04674e6514 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:41.984369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:41.984369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:41.984369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:41.984369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:41.984369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:41.984369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:41.984369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:41.984369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:41.984369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:41.984369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:41.984369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:41.984369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:41.984369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:41.984369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:41.985290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:41.985290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:41.985290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:41.985290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:41.985290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:41.985290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:41.985290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:41.985290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:41.985290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:41.985290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:41.985290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:41.985290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:41.985290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:41.985290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:41.985290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:41.985290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:41.985290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:41.986106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:41.986106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:41.986106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:41.986106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:41.986106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 253/513] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:41.986950 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e182c-b0b3-439a-9e1b-ae04674e6514 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:42.000992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250bed9f-b9de-4f46-a9d3-a850605c84ac None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:42.038824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee2e5c-3c87-4d41-9bd1-c97b581eafea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:42.039391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee2e5c-3c87-4d41-9bd1-c97b581eafea None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:42.039677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee2e5c-3c87-4d41-9bd1-c97b581eafea None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:42.039956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aee2e5c-3c87-4d41-9bd1-c97b581eafea None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:42.040817 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee2e5c-3c87-4d41-9bd1-c97b581eafea None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:42.041132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee2e5c-3c87-4d41-9bd1-c97b581eafea None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:42.042605 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2aee2e5c-3c87-4d41-9bd1-c97b581eafea None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:42.061575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 261/514] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Jul 11 13:53:41 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:42.063010 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee2e5c-3c87-4d41-9bd1-c97b581eafea None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:42.066803 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:42.099233 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309e182c-b0b3-439a-9e1b-ae04674e6514 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:42.137529 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf4d74-f651-4912-9c3f-210e0fd0ec00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:42.138195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf4d74-f651-4912-9c3f-210e0fd0ec00 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:42.138507 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf4d74-f651-4912-9c3f-210e0fd0ec00 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:42.138782 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bf4d74-f651-4912-9c3f-210e0fd0ec00 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:42.139634 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf4d74-f651-4912-9c3f-210e0fd0ec00 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:42.140548 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf4d74-f651-4912-9c3f-210e0fd0ec00 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:42.142000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bf4d74-f651-4912-9c3f-210e0fd0ec00 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:42.174044 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2aee2e5c-3c87-4d41-9bd1-c97b581eafea None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:42.182285 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 254/515] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Jul 11 13:53:41 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:42.183167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf4d74-f651-4912-9c3f-210e0fd0ec00 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:42.208450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:42.214248 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a623f88-9953-4e51-8e57-f242660943fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:42.215105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a623f88-9953-4e51-8e57-f242660943fc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:42.215388 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a623f88-9953-4e51-8e57-f242660943fc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:42.215661 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a623f88-9953-4e51-8e57-f242660943fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:42.222766 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a623f88-9953-4e51-8e57-f242660943fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:42.224213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a623f88-9953-4e51-8e57-f242660943fc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:42.239129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-3a623f88-9953-4e51-8e57-f242660943fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:42.239129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:42.239129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:42.239129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:42.239129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:42.239129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:42.239129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:42.239129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:42.239129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:42.239129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:42.239129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:42.239129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:42.239129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:53:42.239129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:53:42.239129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:53:42.240105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:53:42.240105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:42.240105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:42.240105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:42.240105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:42.240105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:42.240105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:42.240105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:42.240105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:42.240105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:42.240105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:42.240105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:42.240105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:42.240105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:42.240105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:42.240105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:42.240105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:42.240926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:53:42.240926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:53:42.240926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:53:42.240926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:53:42.240926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:53:42.240926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:53:42.240926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:42.240926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:42.240926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 262/516] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:42.241399 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a623f88-9953-4e51-8e57-f242660943fc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:42.290917 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bf4d74-f651-4912-9c3f-210e0fd0ec00 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:42.321351 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:42.327468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b499bb-6d91-456f-9f82-fc4ee5ce6fef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:42.328139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b499bb-6d91-456f-9f82-fc4ee5ce6fef None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:42.328423 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b499bb-6d91-456f-9f82-fc4ee5ce6fef None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:42.328701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b499bb-6d91-456f-9f82-fc4ee5ce6fef None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:42.329436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b499bb-6d91-456f-9f82-fc4ee5ce6fef None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:42.329746 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b499bb-6d91-456f-9f82-fc4ee5ce6fef None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:42.331202 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62b499bb-6d91-456f-9f82-fc4ee5ce6fef None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:42.343445 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 255/517] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:42.344475 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b499bb-6d91-456f-9f82-fc4ee5ce6fef None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:42.347957 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a623f88-9953-4e51-8e57-f242660943fc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:42.383353 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeabd80-65bd-4b99-a42f-2ac5f5514d7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:42.384045 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeabd80-65bd-4b99-a42f-2ac5f5514d7c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:42.384335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeabd80-65bd-4b99-a42f-2ac5f5514d7c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:42.384616 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddeabd80-65bd-4b99-a42f-2ac5f5514d7c None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:42.385351 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeabd80-65bd-4b99-a42f-2ac5f5514d7c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:42.385660 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeabd80-65bd-4b99-a42f-2ac5f5514d7c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:42.387076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddeabd80-65bd-4b99-a42f-2ac5f5514d7c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:42.399277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 263/518] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:42.400142 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeabd80-65bd-4b99-a42f-2ac5f5514d7c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:42.448297 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62b499bb-6d91-456f-9f82-fc4ee5ce6fef None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:42.459680 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:42.481965 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f22f5a-e914-47fe-96b9-3ea003ecf3b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:42.482646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f22f5a-e914-47fe-96b9-3ea003ecf3b8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:42.482928 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f22f5a-e914-47fe-96b9-3ea003ecf3b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:42.483204 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68f22f5a-e914-47fe-96b9-3ea003ecf3b8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:42.507603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddeabd80-65bd-4b99-a42f-2ac5f5514d7c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:42.540769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:42.541665 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24da289-5f6e-47fb-9cc1-1611b5c2a2bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:42.542341 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24da289-5f6e-47fb-9cc1-1611b5c2a2bf None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:42.542655 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24da289-5f6e-47fb-9cc1-1611b5c2a2bf None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:42.542908 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b24da289-5f6e-47fb-9cc1-1611b5c2a2bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:42.552380 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f22f5a-e914-47fe-96b9-3ea003ecf3b8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:42.553845 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68f22f5a-e914-47fe-96b9-3ea003ecf3b8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:42.611694 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24da289-5f6e-47fb-9cc1-1611b5c2a2bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:42.613150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b24da289-5f6e-47fb-9cc1-1611b5c2a2bf None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:42.656473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 256/519] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:42.657333 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f22f5a-e914-47fe-96b9-3ea003ecf3b8 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:42.681827 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:42.735160 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 264/520] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:42.735922 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24da289-5f6e-47fb-9cc1-1611b5c2a2bf None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:42.761814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:42.777199 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68f22f5a-e914-47fe-96b9-3ea003ecf3b8 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:42.834832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e3f62-7281-442f-ab41-a66defd71678 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:42.835949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e3f62-7281-442f-ab41-a66defd71678 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:42.836347 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b24da289-5f6e-47fb-9cc1-1611b5c2a2bf None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:42.836785 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e3f62-7281-442f-ab41-a66defd71678 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:42.836937 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-801e3f62-7281-442f-ab41-a66defd71678 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:42.838482 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e3f62-7281-442f-ab41-a66defd71678 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:42.839031 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e3f62-7281-442f-ab41-a66defd71678 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:42.841714 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-801e3f62-7281-442f-ab41-a66defd71678 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:42.872839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17617a-b0d7-4dac-87ce-70dfac76c08f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:42.873495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17617a-b0d7-4dac-87ce-70dfac76c08f None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:42.873787 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17617a-b0d7-4dac-87ce-70dfac76c08f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:42.874073 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d17617a-b0d7-4dac-87ce-70dfac76c08f None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:42.874860 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d17617a-b0d7-4dac-87ce-70dfac76c08f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:42.880900 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 257/521] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:53:42 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:42.881038 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d17617a-b0d7-4dac-87ce-70dfac76c08f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:42.881114 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e3f62-7281-442f-ab41-a66defd71678 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:42.906442 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 265/522] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 11 13:53:42 2022] POST /identity/v3/users => generated 247 bytes in 171 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:53:42.907560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17617a-b0d7-4dac-87ce-70dfac76c08f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:42.920516 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:43.021332 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-801e3f62-7281-442f-ab41-a66defd71678 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:43.027559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d17617a-b0d7-4dac-87ce-70dfac76c08f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:43.066117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1dbcc3-978f-4fe7-af3d-b51dd537fda1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:43.066597 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1dbcc3-978f-4fe7-af3d-b51dd537fda1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:43.066883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1dbcc3-978f-4fe7-af3d-b51dd537fda1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:43.067166 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef1dbcc3-978f-4fe7-af3d-b51dd537fda1 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:43.070517 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd529a-e1ee-4cd1-b10b-ed994851da7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:43.072437 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd529a-e1ee-4cd1-b10b-ed994851da7d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:43.072728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd529a-e1ee-4cd1-b10b-ed994851da7d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:43.073005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7fd529a-e1ee-4cd1-b10b-ed994851da7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:43.078837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1dbcc3-978f-4fe7-af3d-b51dd537fda1 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:43.080253 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef1dbcc3-978f-4fe7-af3d-b51dd537fda1 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:43.086738 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd529a-e1ee-4cd1-b10b-ed994851da7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:43.088208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7fd529a-e1ee-4cd1-b10b-ed994851da7d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:43.097876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-ef1dbcc3-978f-4fe7-af3d-b51dd537fda1 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 13:53:43.097876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:43.097876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:43.097876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:43.097876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:43.097876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:43.097876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:43.097876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:43.097876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:43.097876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:43.097876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:43.097876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:43.097876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:53:43.097876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:53:43.097876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:53:43.097876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:53:43.099016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:43.099016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:43.099016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:53:43.099016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:43.099016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:53:43.099016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:43.099016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:43.099016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:43.099016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:43.099016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:43.099016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:43.099016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:43.099016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:43.099016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:43.099016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:43.099016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:43.099016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:43.099016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:43.100252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:43.100252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:43.100252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:53:43.100252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:53:43.100252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:53:43.100252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:53:43.100252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:53:43.100252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:53:43.100252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 13:53:43.100252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:43.102145 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 258/523] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:53:42 2022] GET /identity/v3/users/neutron => generated 85 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:43.103255 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1dbcc3-978f-4fe7-af3d-b51dd537fda1 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:43.115100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-f7fd529a-e1ee-4cd1-b10b-ed994851da7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:43.115100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:43.115100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:43.115100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:43.115100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:43.115100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:43.115100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:43.115100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:43.115100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:43.115100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:43.115100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:43.115100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:43.115100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:43.115100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:43.115100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:43.116306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:43.116306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:43.116306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:43.116306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:43.116306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:43.116306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:43.116306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:43.116306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:43.116306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:43.116306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:43.116306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:43.116306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:43.116306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:43.116306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:43.116306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:43.116306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:43.116306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:43.117384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:43.117384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:43.117384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:43.117384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:43.117384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 266/524] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:43.117384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd529a-e1ee-4cd1-b10b-ed994851da7d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:43.123627 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:43.199454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:43.240130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef1dbcc3-978f-4fe7-af3d-b51dd537fda1 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:43.257809 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7fd529a-e1ee-4cd1-b10b-ed994851da7d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:43.289538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be97766-86ec-415a-8b74-5eae8d1500fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:43.290255 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be97766-86ec-415a-8b74-5eae8d1500fd None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:43.291198 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be97766-86ec-415a-8b74-5eae8d1500fd None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:43.291198 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be97766-86ec-415a-8b74-5eae8d1500fd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:43.301399 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f7a15-a015-4723-86e1-e16e2e078335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:43.301838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be97766-86ec-415a-8b74-5eae8d1500fd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:43.302108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f7a15-a015-4723-86e1-e16e2e078335 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:43.302378 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f7a15-a015-4723-86e1-e16e2e078335 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:43.302659 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f7a15-a015-4723-86e1-e16e2e078335 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:43.303349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be97766-86ec-415a-8b74-5eae8d1500fd None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:43.303650 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f7a15-a015-4723-86e1-e16e2e078335 None admin] RBAC: Adding query filter params (role.id=1f7cf26664d04e5e9331ead293476442, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=ef5dac3f311e43809a46995bab56e780) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:43.303925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f7a15-a015-4723-86e1-e16e2e078335 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:43.305783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f7a15-a015-4723-86e1-e16e2e078335 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:43.312430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 267/525] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:53:43 2022] GET /identity/v3/role_assignments?user.id=ef5dac3f311e43809a46995bab56e780&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=1f7cf26664d04e5e9331ead293476442 => generated 259 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:43.313116 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f7a15-a015-4723-86e1-e16e2e078335 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:43.338468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-6be97766-86ec-415a-8b74-5eae8d1500fd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:43.338468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:43.338468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:43.338468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:43.338468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:43.338468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:43.338468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:43.338468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:43.338468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:43.338468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:43.338468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:43.338468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:43.338468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:43.338468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:43.338468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:43.339442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:43.339442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:43.339442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:43.339442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:43.339442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:43.339442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:43.339442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:43.339442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:43.339442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:43.339442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:43.339442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:43.339442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:43.339442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:43.339442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:43.339442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:43.339442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:43.339442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:43.340276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:43.340276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:43.340276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:43.340276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:43.344298 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 259/526] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:43.345770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be97766-86ec-415a-8b74-5eae8d1500fd None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:43.444399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:43.484385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f7a15-a015-4723-86e1-e16e2e078335 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:43.526760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:43.535123 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6764b0f-5b3d-4bd3-b249-5efe676e6374 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:43.535933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6764b0f-5b3d-4bd3-b249-5efe676e6374 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:43.536296 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6764b0f-5b3d-4bd3-b249-5efe676e6374 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:43.536649 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6764b0f-5b3d-4bd3-b249-5efe676e6374 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:43.536951 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be97766-86ec-415a-8b74-5eae8d1500fd None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:43.537671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6764b0f-5b3d-4bd3-b249-5efe676e6374 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:43.538092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6764b0f-5b3d-4bd3-b249-5efe676e6374 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:43.539757 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6764b0f-5b3d-4bd3-b249-5efe676e6374 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:43.564280 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 268/527] 10.222.0.49 () {56 vars in 1139 bytes} [Mon Jul 11 13:53:43 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:43.565255 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6764b0f-5b3d-4bd3-b249-5efe676e6374 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:43.613974 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85d8c5-9ced-4545-95a9-795770a54b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:43.615258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85d8c5-9ced-4545-95a9-795770a54b95 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:43.615622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85d8c5-9ced-4545-95a9-795770a54b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:43.616158 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b85d8c5-9ced-4545-95a9-795770a54b95 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:43.617055 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85d8c5-9ced-4545-95a9-795770a54b95 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:43.617374 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85d8c5-9ced-4545-95a9-795770a54b95 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:43.618906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b85d8c5-9ced-4545-95a9-795770a54b95 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:43.637937 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 260/528] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:43.638817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85d8c5-9ced-4545-95a9-795770a54b95 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:43.705877 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6764b0f-5b3d-4bd3-b249-5efe676e6374 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:43.743105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3d7e8-6cc8-42dd-8a6e-0eb41fadcb32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:43.744250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3d7e8-6cc8-42dd-8a6e-0eb41fadcb32 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:43.744734 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3d7e8-6cc8-42dd-8a6e-0eb41fadcb32 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:43.745207 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fe3d7e8-6cc8-42dd-8a6e-0eb41fadcb32 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:43.746160 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3d7e8-6cc8-42dd-8a6e-0eb41fadcb32 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:43.746701 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3d7e8-6cc8-42dd-8a6e-0eb41fadcb32 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:43.748301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fe3d7e8-6cc8-42dd-8a6e-0eb41fadcb32 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:43.759621 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b85d8c5-9ced-4545-95a9-795770a54b95 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:43.762193 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 269/529] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:43.763402 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3d7e8-6cc8-42dd-8a6e-0eb41fadcb32 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:43.786672 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:43.805380 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f3e88-5471-4f05-ac80-c36546973449 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:43.807271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f3e88-5471-4f05-ac80-c36546973449 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:43.807713 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f3e88-5471-4f05-ac80-c36546973449 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:43.808131 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43f3e88-5471-4f05-ac80-c36546973449 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:43.817345 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:43.895627 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43f3e88-5471-4f05-ac80-c36546973449 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:43.897663 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c43f3e88-5471-4f05-ac80-c36546973449 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:43.901528 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fe3d7e8-6cc8-42dd-8a6e-0eb41fadcb32 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:43.960566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64befdfc-9903-4655-83d7-7b2414254f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:43.962374 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64befdfc-9903-4655-83d7-7b2414254f74 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:43.962912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64befdfc-9903-4655-83d7-7b2414254f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:43.963357 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64befdfc-9903-4655-83d7-7b2414254f74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:43.973970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64befdfc-9903-4655-83d7-7b2414254f74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:43.976702 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64befdfc-9903-4655-83d7-7b2414254f74 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:43.996693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-64befdfc-9903-4655-83d7-7b2414254f74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:43.996693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:43.996693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:43.996693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:43.996693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:43.996693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:43.996693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:43.996693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:43.996693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:43.996693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:43.996693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:43.996693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:43.996693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:53:43.996693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:53:43.996693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:53:43.997783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:53:43.997783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:43.997783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:43.997783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:43.997783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:43.997783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:43.997783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:43.997783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:43.997783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:43.997783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:43.997783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:43.997783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:43.997783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:43.997783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:43.997783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:43.997783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:43.997783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:43.998628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:53:43.998628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:53:43.998628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:53:43.998628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:53:43.998628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:53:43.998628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:53:43.998628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:43.998628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:44.005318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 270/530] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:43 2022] GET /identity/v3/projects/service => generated 88 bytes in 242 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:44.005766 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64befdfc-9903-4655-83d7-7b2414254f74 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:44.052868 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 261/531] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:44.052868 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f3e88-5471-4f05-ac80-c36546973449 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:44.087453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:44.140778 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64befdfc-9903-4655-83d7-7b2414254f74 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:44.142822 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:44.179287 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43f3e88-5471-4f05-ac80-c36546973449 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:44.181922 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3006b-352e-49aa-a0de-ba998ca26a1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:44.182985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3006b-352e-49aa-a0de-ba998ca26a1b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:44.183491 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3006b-352e-49aa-a0de-ba998ca26a1b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:44.183982 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd3006b-352e-49aa-a0de-ba998ca26a1b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:44.191768 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3006b-352e-49aa-a0de-ba998ca26a1b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:44.193587 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd3006b-352e-49aa-a0de-ba998ca26a1b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:44.208927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-abd3006b-352e-49aa-a0de-ba998ca26a1b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:44.208927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:44.208927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:44.208927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:44.208927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:44.208927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:44.208927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:44.208927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:44.208927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:44.208927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:44.208927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:44.208927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:44.208927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:53:44.208927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:53:44.208927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:53:44.209853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:53:44.209853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:44.209853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:44.209853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:44.209853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:44.209853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:44.209853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:44.209853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:44.209853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:44.209853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:44.209853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:44.209853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:44.209853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:44.209853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:44.209853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:44.209853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:44.209853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:44.210687 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:53:44.210687 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:53:44.210687 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:53:44.210687 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:53:44.210687 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:53:44.210687 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:53:44.210687 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:44.210687 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:44.213150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 271/532] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:44.214179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3006b-352e-49aa-a0de-ba998ca26a1b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:44.222625 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3eb03-92cf-478e-af1b-a3a225615789 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:44.223303 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3eb03-92cf-478e-af1b-a3a225615789 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:44.223597 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3eb03-92cf-478e-af1b-a3a225615789 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:44.223877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69f3eb03-92cf-478e-af1b-a3a225615789 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:44.224685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3eb03-92cf-478e-af1b-a3a225615789 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:44.224998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3eb03-92cf-478e-af1b-a3a225615789 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:44.226671 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69f3eb03-92cf-478e-af1b-a3a225615789 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:44.239835 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 262/533] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:53:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:44.244715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3eb03-92cf-478e-af1b-a3a225615789 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:44.346252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd3006b-352e-49aa-a0de-ba998ca26a1b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:44.348686 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:44.370601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69f3eb03-92cf-478e-af1b-a3a225615789 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:44.383169 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec973aaa-73e6-4260-b1a5-23ea258fd4ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:44.383961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec973aaa-73e6-4260-b1a5-23ea258fd4ad None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:44.384397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec973aaa-73e6-4260-b1a5-23ea258fd4ad None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:44.384687 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec973aaa-73e6-4260-b1a5-23ea258fd4ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:44.396488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec973aaa-73e6-4260-b1a5-23ea258fd4ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:44.398112 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec973aaa-73e6-4260-b1a5-23ea258fd4ad None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:44.411066 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c157499-41a4-4734-9c4c-9d09014e4988 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:44.412075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c157499-41a4-4734-9c4c-9d09014e4988 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:44.412533 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c157499-41a4-4734-9c4c-9d09014e4988 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:44.412998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c157499-41a4-4734-9c4c-9d09014e4988 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:44.414010 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c157499-41a4-4734-9c4c-9d09014e4988 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:44.414550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c157499-41a4-4734-9c4c-9d09014e4988 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:44.416209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c157499-41a4-4734-9c4c-9d09014e4988 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:44.417108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-ec973aaa-73e6-4260-b1a5-23ea258fd4ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:44.417108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:44.417108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:44.417108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:44.417108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:44.417108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:44.417108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:44.417108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:44.417108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:44.417108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:44.417108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:44.417108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:44.417108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:44.417108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:44.417108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:44.417965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:44.417965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:44.417965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:44.417965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:44.417965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:44.417965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:44.417965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:44.417965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:44.417965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:44.417965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:44.417965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:44.417965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:44.417965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:44.417965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:44.417965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:44.417965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:44.417965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:44.418839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:44.418839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:44.418839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:44.418839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:44.418839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 272/534] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:44.419951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec973aaa-73e6-4260-b1a5-23ea258fd4ad None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:44.434361 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 263/535] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:53:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:44.435459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c157499-41a4-4734-9c4c-9d09014e4988 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:44.444307 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:44.537883 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec973aaa-73e6-4260-b1a5-23ea258fd4ad None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:44.555156 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c157499-41a4-4734-9c4c-9d09014e4988 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:44.575614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e9f4c-b66c-4759-9c3c-b5149fb6f220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:44.576300 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e9f4c-b66c-4759-9c3c-b5149fb6f220 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:44.576694 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e9f4c-b66c-4759-9c3c-b5149fb6f220 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:44.577096 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-594e9f4c-b66c-4759-9c3c-b5149fb6f220 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:44.578686 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e9f4c-b66c-4759-9c3c-b5149fb6f220 None admin] RBAC: Adding query filter params (role.id=1f7cf26664d04e5e9331ead293476442, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=6bb1616f632d4c67828cce3fc41945b7) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:44.579110 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e9f4c-b66c-4759-9c3c-b5149fb6f220 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:44.580901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-594e9f4c-b66c-4759-9c3c-b5149fb6f220 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:44.594335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 273/536] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:53:44 2022] GET /identity/v3/role_assignments?user.id=6bb1616f632d4c67828cce3fc41945b7&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=1f7cf26664d04e5e9331ead293476442 => generated 259 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:44.594335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e9f4c-b66c-4759-9c3c-b5149fb6f220 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:44.618368 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9d9b3-d712-4c32-a7da-95d17617bf7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:44.619015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9d9b3-d712-4c32-a7da-95d17617bf7a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:44.619305 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9d9b3-d712-4c32-a7da-95d17617bf7a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:44.619586 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f9d9b3-d712-4c32-a7da-95d17617bf7a None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:44.620354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9d9b3-d712-4c32-a7da-95d17617bf7a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:44.620668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9d9b3-d712-4c32-a7da-95d17617bf7a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:44.625077 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f9d9b3-d712-4c32-a7da-95d17617bf7a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:44.655405 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 264/537] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:44.660505 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9d9b3-d712-4c32-a7da-95d17617bf7a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:44.666472 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:44.756139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:44.813466 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f9d9b3-d712-4c32-a7da-95d17617bf7a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:44.839146 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-594e9f4c-b66c-4759-9c3c-b5149fb6f220 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:44.868543 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c665c9-09ae-47ce-bbe7-ca99b46038f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:44.869807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c665c9-09ae-47ce-bbe7-ca99b46038f1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:44.870294 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c665c9-09ae-47ce-bbe7-ca99b46038f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:44.870825 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c665c9-09ae-47ce-bbe7-ca99b46038f1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:44.886966 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278e5b4-9165-448d-88f9-ed5b27d93134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:44.890344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278e5b4-9165-448d-88f9-ed5b27d93134 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:44.890344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278e5b4-9165-448d-88f9-ed5b27d93134 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:44.890344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2278e5b4-9165-448d-88f9-ed5b27d93134 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:44.890344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278e5b4-9165-448d-88f9-ed5b27d93134 None admin] RBAC: Adding query filter params (role.id=1f7cf26664d04e5e9331ead293476442, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=f80d59fec4994e689e857ba2087d3a23) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:44.890344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278e5b4-9165-448d-88f9-ed5b27d93134 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:44.891041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2278e5b4-9165-448d-88f9-ed5b27d93134 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:44.892030 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c665c9-09ae-47ce-bbe7-ca99b46038f1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:44.893836 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3c665c9-09ae-47ce-bbe7-ca99b46038f1 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:44.898157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 274/538] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:53:44 2022] GET /identity/v3/role_assignments?user.id=f80d59fec4994e689e857ba2087d3a23&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=1f7cf26664d04e5e9331ead293476442 => generated 259 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:44.925960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-f3c665c9-09ae-47ce-bbe7-ca99b46038f1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:44.925960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:44.925960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:44.925960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:44.925960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:44.925960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:44.925960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:44.925960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:44.925960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:44.925960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:44.925960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:44.925960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:44.925960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:53:44.925960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:53:44.925960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:53:44.926906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:53:44.926906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:44.926906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:44.926906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:44.926906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:44.926906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:44.926906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:44.926906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:44.926906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:44.926906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:44.926906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:44.926906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:44.926906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:44.926906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:44.926906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:44.926906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:44.926906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:44.927745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:53:44.927745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:53:44.927745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:53:44.927745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:53:44.927745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:53:44.927745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:53:44.927745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:44.927745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:44.930845 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 265/539] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:44 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:44.938420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278e5b4-9165-448d-88f9-ed5b27d93134 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:45.087557 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2278e5b4-9165-448d-88f9-ed5b27d93134 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:45.127967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ae0aab-5998-4fb6-8dde-05d05336a151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:45.128578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae0aab-5998-4fb6-8dde-05d05336a151 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:45.128872 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae0aab-5998-4fb6-8dde-05d05336a151 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:45.129154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ae0aab-5998-4fb6-8dde-05d05336a151 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:45.129983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae0aab-5998-4fb6-8dde-05d05336a151 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:45.130326 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae0aab-5998-4fb6-8dde-05d05336a151 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:45.131883 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ae0aab-5998-4fb6-8dde-05d05336a151 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:45.152909 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 275/540] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:53:44 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:45.159387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c665c9-09ae-47ce-bbe7-ca99b46038f1 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:45.262736 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:45.274850 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e557bd8-20ce-4fff-b05b-849dc06e6a3e None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:45.275161 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e557bd8-20ce-4fff-b05b-849dc06e6a3e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:45.275421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e557bd8-20ce-4fff-b05b-849dc06e6a3e None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:45.277146 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 276/541] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:45.283248 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8f433-624c-4ec0-a782-0315f099e7dd None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:45.283536 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8f433-624c-4ec0-a782-0315f099e7dd None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:45.283821 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda8f433-624c-4ec0-a782-0315f099e7dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:45.289180 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:45.303117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-dda8f433-624c-4ec0-a782-0315f099e7dd None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:45.304519 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dda8f433-624c-4ec0-a782-0315f099e7dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:45.347338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c665c9-09ae-47ce-bbe7-ca99b46038f1 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:45.386021 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 277/542] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:53:45.413851 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8f433-624c-4ec0-a782-0315f099e7dd None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:45.423178 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c51a41-ffa4-4422-98ae-0bed4feb1d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:45.424098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c51a41-ffa4-4422-98ae-0bed4feb1d7e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:45.424555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c51a41-ffa4-4422-98ae-0bed4feb1d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:45.424984 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c51a41-ffa4-4422-98ae-0bed4feb1d7e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:45.426115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c51a41-ffa4-4422-98ae-0bed4feb1d7e None admin] RBAC: Adding query filter params (role.id=1f7cf26664d04e5e9331ead293476442, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=ce1c85ee44ef45d2b631e51112b5ea0a) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:45.426618 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c51a41-ffa4-4422-98ae-0bed4feb1d7e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:45.428387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c51a41-ffa4-4422-98ae-0bed4feb1d7e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:45.439009 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 266/543] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:53:45 2022] GET /identity/v3/role_assignments?user.id=ce1c85ee44ef45d2b631e51112b5ea0a&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=1f7cf26664d04e5e9331ead293476442 => generated 259 bytes in 280 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:45.531466 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda8f433-624c-4ec0-a782-0315f099e7dd None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:45.566900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:45.571459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29a7f4d-bc22-4c4c-b1d0-f36b1be5b439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:45.572253 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a7f4d-bc22-4c4c-b1d0-f36b1be5b439 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:45.572549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a7f4d-bc22-4c4c-b1d0-f36b1be5b439 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:45.572820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29a7f4d-bc22-4c4c-b1d0-f36b1be5b439 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:45.586777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29a7f4d-bc22-4c4c-b1d0-f36b1be5b439 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:45.595417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a29a7f4d-bc22-4c4c-b1d0-f36b1be5b439 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:45.595417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:45.595417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:45.595417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:45.595417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:45.595417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:45.595417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:45.595417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:45.595417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:45.595417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:45.595417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:45.595417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:45.595417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:45.595417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:45.595417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:45.595417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:45.596361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:45.596361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:45.596361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:45.596361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:45.596361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:45.596361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:45.596361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:45.596361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:45.596361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:45.596361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:45.596361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:45.596361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:45.596361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:45.596361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:45.596361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:45.596361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:45.597147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:45.597147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:45.597147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:45.597147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:45.597147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:45.597147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:45.597147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:45.597147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:45.597147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:45.597147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:45.597147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:45.597147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:45.597147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 278/544] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:53:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:53:45.603042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c51a41-ffa4-4422-98ae-0bed4feb1d7e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:45.714146 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70acc30-e64a-449b-b5e7-45c56dafb7e5 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:45.714146 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70acc30-e64a-449b-b5e7-45c56dafb7e5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:45.714146 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70acc30-e64a-449b-b5e7-45c56dafb7e5 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:45.715989 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 279/545] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:53:45.721047 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb70fc0-fd4e-4dc8-b33a-c67831547273 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:45.721381 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb70fc0-fd4e-4dc8-b33a-c67831547273 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:45.721650 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb70fc0-fd4e-4dc8-b33a-c67831547273 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:45.751194 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9bb70fc0-fd4e-4dc8-b33a-c67831547273 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:45.752443 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9bb70fc0-fd4e-4dc8-b33a-c67831547273 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:45.823835 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c51a41-ffa4-4422-98ae-0bed4feb1d7e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:45.857658 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:45.887206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3ccb9-af01-4fe6-aa2d-e1d389eaf8c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:45.888336 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3ccb9-af01-4fe6-aa2d-e1d389eaf8c6 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:45.888835 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3ccb9-af01-4fe6-aa2d-e1d389eaf8c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:45.889337 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab3ccb9-af01-4fe6-aa2d-e1d389eaf8c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:45.890917 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3ccb9-af01-4fe6-aa2d-e1d389eaf8c6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:45.891473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3ccb9-af01-4fe6-aa2d-e1d389eaf8c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:45.893147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab3ccb9-af01-4fe6-aa2d-e1d389eaf8c6 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:45.911555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 267/546] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:53:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 309 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:45.918717 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3ccb9-af01-4fe6-aa2d-e1d389eaf8c6 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:45.961714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 280/547] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 241 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:53:45.990048 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb70fc0-fd4e-4dc8-b33a-c67831547273 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:46.065094 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab3ccb9-af01-4fe6-aa2d-e1d389eaf8c6 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:46.123099 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d59e08-e7e2-4ea8-9c0c-4c982f1f1743 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:46.123960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d59e08-e7e2-4ea8-9c0c-4c982f1f1743 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:46.124762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d59e08-e7e2-4ea8-9c0c-4c982f1f1743 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:46.125047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3d59e08-e7e2-4ea8-9c0c-4c982f1f1743 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:46.145782 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d59e08-e7e2-4ea8-9c0c-4c982f1f1743 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:46.163518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3d59e08-e7e2-4ea8-9c0c-4c982f1f1743 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:46.184940 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb70fc0-fd4e-4dc8-b33a-c67831547273 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:46.202701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-b3d59e08-e7e2-4ea8-9c0c-4c982f1f1743 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:46.202701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:46.202701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:46.202701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:46.202701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:46.202701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:46.202701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:46.202701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:46.202701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:46.202701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:46.202701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:46.202701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:46.202701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:46.202701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:46.202701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:46.203628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:46.203628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:46.203628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:46.203628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:46.203628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:46.203628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:46.203628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:46.203628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:46.203628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:46.203628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:46.203628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:46.203628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:46.203628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:46.203628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:46.203628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:46.203628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:46.203628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:46.204482 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:46.204482 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:46.204482 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:46.204482 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:46.210508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 268/548] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:53:46.211804 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d59e08-e7e2-4ea8-9c0c-4c982f1f1743 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:46.213437 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:46.231904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12553e5d-30ca-4112-80f7-247da8254c0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:46.234452 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12553e5d-30ca-4112-80f7-247da8254c0a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:46.235144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12553e5d-30ca-4112-80f7-247da8254c0a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:46.236871 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12553e5d-30ca-4112-80f7-247da8254c0a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:46.263630 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12553e5d-30ca-4112-80f7-247da8254c0a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:46.278737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-12553e5d-30ca-4112-80f7-247da8254c0a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:46.278737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:46.278737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:46.278737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:46.278737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:46.278737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:46.278737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:46.278737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:46.278737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:46.278737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:46.278737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:46.278737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:46.278737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:46.278737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:46.278737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:46.278737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:46.279743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:46.279743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:46.279743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:46.279743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:46.279743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:46.279743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:46.279743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:46.279743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:46.279743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:46.279743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:46.279743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:46.279743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:46.279743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:46.279743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:46.279743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:46.279743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:46.281099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:46.281099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:46.281099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:46.281099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:46.281099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:46.281099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:46.281099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:46.281099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:46.281099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:46.281099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:46.281099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:46.281099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:46.288738 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 281/549] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:53:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:46.298882 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12553e5d-30ca-4112-80f7-247da8254c0a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:46.309152 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:46.429824 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3d59e08-e7e2-4ea8-9c0c-4c982f1f1743 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:46.492150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd6bd3-f32c-474c-af5b-d90bc906f30e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:46.493601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd6bd3-f32c-474c-af5b-d90bc906f30e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:46.494276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd6bd3-f32c-474c-af5b-d90bc906f30e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:46.495242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cd6bd3-f32c-474c-af5b-d90bc906f30e None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:46.497195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd6bd3-f32c-474c-af5b-d90bc906f30e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:46.498069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd6bd3-f32c-474c-af5b-d90bc906f30e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:46.501158 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cd6bd3-f32c-474c-af5b-d90bc906f30e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:46.509602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12553e5d-30ca-4112-80f7-247da8254c0a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:46.544584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 269/550] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:46.549644 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78527d7d-a954-4c2c-8745-eef3ee23e8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:46.550349 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78527d7d-a954-4c2c-8745-eef3ee23e8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:46.550640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78527d7d-a954-4c2c-8745-eef3ee23e8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:46.550917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78527d7d-a954-4c2c-8745-eef3ee23e8d0 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:46.551690 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78527d7d-a954-4c2c-8745-eef3ee23e8d0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:46.551995 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78527d7d-a954-4c2c-8745-eef3ee23e8d0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:46.552682 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd6bd3-f32c-474c-af5b-d90bc906f30e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:46.553514 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78527d7d-a954-4c2c-8745-eef3ee23e8d0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:46.563749 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 282/551] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:53:46 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:46.578396 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78527d7d-a954-4c2c-8745-eef3ee23e8d0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:46.642323 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:46.754562 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78527d7d-a954-4c2c-8745-eef3ee23e8d0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:46.777831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cd6bd3-f32c-474c-af5b-d90bc906f30e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:46.790769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:46.801293 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22239d77-5c29-47bc-b1b9-0f36c8f2f5ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:46.802018 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22239d77-5c29-47bc-b1b9-0f36c8f2f5ca None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:46.802365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22239d77-5c29-47bc-b1b9-0f36c8f2f5ca None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:46.802646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22239d77-5c29-47bc-b1b9-0f36c8f2f5ca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:46.818861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22239d77-5c29-47bc-b1b9-0f36c8f2f5ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:46.820339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22239d77-5c29-47bc-b1b9-0f36c8f2f5ca None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:46.848692 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b458b-f509-4264-88d3-422b1ffb482b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:46.849501 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b458b-f509-4264-88d3-422b1ffb482b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:46.849782 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b458b-f509-4264-88d3-422b1ffb482b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:46.850057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6b458b-f509-4264-88d3-422b1ffb482b None admin] PATH_INFO: `/v3/users/swift` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:46.855708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-22239d77-5c29-47bc-b1b9-0f36c8f2f5ca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:46.855708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:46.855708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:46.855708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:46.855708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:46.855708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:46.855708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:46.855708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:46.855708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:46.855708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:46.855708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:46.855708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:46.855708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:46.855708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:46.855708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:46.856723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:46.856723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:46.856723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:46.856723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:46.856723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:46.856723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:46.856723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:46.856723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:46.856723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:46.856723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:46.856723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:46.856723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:46.856723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:46.856723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:46.856723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:46.856723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:46.856723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:46.857627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:46.857627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:46.857627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:46.857627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:46.863775 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 283/552] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:46.865577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22239d77-5c29-47bc-b1b9-0f36c8f2f5ca None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:46.866656 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b458b-f509-4264-88d3-422b1ffb482b None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:46.868133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6b458b-f509-4264-88d3-422b1ffb482b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:46.898762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-9a6b458b-f509-4264-88d3-422b1ffb482b None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 13:53:46.898762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:46.898762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:46.898762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:46.898762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:46.898762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:46.898762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:46.898762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:46.898762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:46.898762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:46.898762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:46.898762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:46.898762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:53:46.898762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:53:46.898762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:53:46.898762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:53:46.899758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:46.899758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:46.899758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:53:46.899758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:46.899758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:53:46.899758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:46.899758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:46.899758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:46.899758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:46.899758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:46.899758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:46.899758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:46.899758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:46.899758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:46.899758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:46.899758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:46.899758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:46.899758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:46.900681 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:46.900681 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:46.900681 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:53:46.900681 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:53:46.900681 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:53:46.900681 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:53:46.900681 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:53:46.900681 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:53:46.900681 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 13:53:46.900681 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:46.907739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 270/553] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:53:46 2022] GET /identity/v3/users/swift => generated 83 bytes in 358 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:46.913516 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b458b-f509-4264-88d3-422b1ffb482b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:46.996289 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22239d77-5c29-47bc-b1b9-0f36c8f2f5ca None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:46.998060 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:47.034735 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101dbe83-b2d9-4be4-9112-f52a6c898b37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:47.035412 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101dbe83-b2d9-4be4-9112-f52a6c898b37 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:47.035699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101dbe83-b2d9-4be4-9112-f52a6c898b37 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:47.035978 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101dbe83-b2d9-4be4-9112-f52a6c898b37 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:47.036714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101dbe83-b2d9-4be4-9112-f52a6c898b37 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:47.037025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101dbe83-b2d9-4be4-9112-f52a6c898b37 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:47.038420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101dbe83-b2d9-4be4-9112-f52a6c898b37 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:47.055483 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 284/554] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:47.059718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101dbe83-b2d9-4be4-9112-f52a6c898b37 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:47.151624 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6b458b-f509-4264-88d3-422b1ffb482b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:47.195348 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0632af-a0a5-40ae-93cb-26d0680a0d35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:47.196029 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0632af-a0a5-40ae-93cb-26d0680a0d35 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:47.196374 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0632af-a0a5-40ae-93cb-26d0680a0d35 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:47.196646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0632af-a0a5-40ae-93cb-26d0680a0d35 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:47.197706 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0632af-a0a5-40ae-93cb-26d0680a0d35 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:47.198127 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0632af-a0a5-40ae-93cb-26d0680a0d35 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:47.199851 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0632af-a0a5-40ae-93cb-26d0680a0d35 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:47.221043 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 271/555] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Jul 11 13:53:46 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:47.222288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101dbe83-b2d9-4be4-9112-f52a6c898b37 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:47.222421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665ee9a-165d-49a1-a1dd-2ab2084f7bdc None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:47.222705 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665ee9a-165d-49a1-a1dd-2ab2084f7bdc None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:47.222981 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4665ee9a-165d-49a1-a1dd-2ab2084f7bdc None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:47.224601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 272/556] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:47.226317 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665ee9a-165d-49a1-a1dd-2ab2084f7bdc None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:47.248815 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:47.268428 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbfd4e2-2f78-4dd4-9bed-cfddbe6b389d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:47.269480 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfd4e2-2f78-4dd4-9bed-cfddbe6b389d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:47.270066 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfd4e2-2f78-4dd4-9bed-cfddbe6b389d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:47.270552 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbfd4e2-2f78-4dd4-9bed-cfddbe6b389d None admin] PATH_INFO: `/v3/users/nova` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:47.281897 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbfd4e2-2f78-4dd4-9bed-cfddbe6b389d None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:47.283790 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bbfd4e2-2f78-4dd4-9bed-cfddbe6b389d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:47.303498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-4bbfd4e2-2f78-4dd4-9bed-cfddbe6b389d None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 13:53:47.303498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:47.303498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:47.303498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:47.303498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:47.303498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:47.303498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:47.303498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:47.303498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:47.303498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:47.303498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:47.303498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:47.303498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:53:47.303498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:53:47.303498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:53:47.303498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:53:47.304772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:47.304772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:47.304772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:53:47.304772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:47.304772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:53:47.304772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:47.304772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:47.304772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:47.304772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:47.304772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:47.304772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:47.304772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:47.304772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:47.304772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:47.304772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:47.304772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:47.304772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:47.304772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:47.306722 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:47.306722 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:47.306722 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:53:47.306722 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:53:47.306722 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:53:47.306722 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:53:47.306722 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:53:47.306722 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:53:47.306722 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 13:53:47.306722 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:47.315067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 285/557] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Jul 11 13:53:47 2022] GET /identity/v3/users/nova => generated 82 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:47.315067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056eb250-4b61-43ab-9669-0986b5e23320 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:47.315067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056eb250-4b61-43ab-9669-0986b5e23320 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:47.315067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056eb250-4b61-43ab-9669-0986b5e23320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:47.339718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-056eb250-4b61-43ab-9669-0986b5e23320 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:47.341332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-056eb250-4b61-43ab-9669-0986b5e23320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:47.355343 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:47.365995 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4665ee9a-165d-49a1-a1dd-2ab2084f7bdc None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:47.413266 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f8866f-170f-4c37-9adc-6e1488bff9c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:47.414745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8866f-170f-4c37-9adc-6e1488bff9c2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:47.415489 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8866f-170f-4c37-9adc-6e1488bff9c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:47.415960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f8866f-170f-4c37-9adc-6e1488bff9c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:47.431480 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 286/558] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:47.431480 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056eb250-4b61-43ab-9669-0986b5e23320 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:47.521780 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8866f-170f-4c37-9adc-6e1488bff9c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:47.524801 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f8866f-170f-4c37-9adc-6e1488bff9c2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:47.550026 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:47.573899 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056eb250-4b61-43ab-9669-0986b5e23320 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:47.616127 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805b78f-19f6-4d85-8f96-2872e1dcbc07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:47.617220 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805b78f-19f6-4d85-8f96-2872e1dcbc07 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:47.617220 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805b78f-19f6-4d85-8f96-2872e1dcbc07 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:47.617496 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2805b78f-19f6-4d85-8f96-2872e1dcbc07 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:47.618292 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805b78f-19f6-4d85-8f96-2872e1dcbc07 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:47.618627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805b78f-19f6-4d85-8f96-2872e1dcbc07 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:47.620044 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2805b78f-19f6-4d85-8f96-2872e1dcbc07 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:47.641460 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 287/559] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Jul 11 13:53:47 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:47.642114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805b78f-19f6-4d85-8f96-2872e1dcbc07 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:47.646712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:47.658585 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 273/560] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:47 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 433 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:47.660002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f3268-c966-452b-8a02-6efa4e2202e4 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:47.660302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f3268-c966-452b-8a02-6efa4e2202e4 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:47.660588 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71f3268-c966-452b-8a02-6efa4e2202e4 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:47.666180 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 274/561] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:47.666180 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f3268-c966-452b-8a02-6efa4e2202e4 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:47.776346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2805b78f-19f6-4d85-8f96-2872e1dcbc07 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:47.791807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71f3268-c966-452b-8a02-6efa4e2202e4 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:47.813289 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:47.820025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cafdd-e90a-47b0-9ba6-4a77e0e34812 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:47.820996 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cafdd-e90a-47b0-9ba6-4a77e0e34812 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:47.821288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cafdd-e90a-47b0-9ba6-4a77e0e34812 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:47.821569 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71cafdd-e90a-47b0-9ba6-4a77e0e34812 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:47.833531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5248079b-4cd5-497c-adeb-f82186090f1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:47.833715 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71cafdd-e90a-47b0-9ba6-4a77e0e34812 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:47.834751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248079b-4cd5-497c-adeb-f82186090f1e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:47.835218 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248079b-4cd5-497c-adeb-f82186090f1e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:47.835691 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5248079b-4cd5-497c-adeb-f82186090f1e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:47.841314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c71cafdd-e90a-47b0-9ba6-4a77e0e34812 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:47.841314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:47.841314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:47.841314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:47.841314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:47.841314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:47.841314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:47.841314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:47.841314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:47.841314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:47.841314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:47.841314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:47.841314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:47.841314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:47.841314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:47.841314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:47.842252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:47.842252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:47.842252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:47.842252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:47.842252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:47.842252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:47.842252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:47.842252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:47.842252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:47.842252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:47.842252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:47.842252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:47.842252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:47.842252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:47.842252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:47.842252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:47.843208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:47.843208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:47.843208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:47.843208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:47.843208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:47.843208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:47.843208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:47.843208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:47.843208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:47.843208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:47.843208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:47.843208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:47.843208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 288/562] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:53:47 2022] GET /identity/v3/roles/service => generated 85 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:47.843954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cafdd-e90a-47b0-9ba6-4a77e0e34812 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:47.853714 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248079b-4cd5-497c-adeb-f82186090f1e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:47.855848 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5248079b-4cd5-497c-adeb-f82186090f1e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:47.877855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-5248079b-4cd5-497c-adeb-f82186090f1e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:47.877855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:47.877855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:47.877855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:47.877855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:47.877855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:47.877855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:47.877855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:47.877855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:47.877855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:47.877855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:47.877855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:47.877855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:47.877855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:47.877855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:47.879306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:47.879306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:47.879306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:47.879306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:47.879306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:47.879306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:47.879306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:47.879306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:47.879306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:47.879306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:47.879306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:47.879306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:47.879306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:47.879306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:47.879306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:47.879306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:47.879306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:47.880665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:47.880665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:47.880665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:47.880665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:47.880665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 275/563] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:47.880665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd59ec8-9a8e-4ab6-830b-6052efbdb5e8 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:47.881011 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd59ec8-9a8e-4ab6-830b-6052efbdb5e8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:47.881818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acd59ec8-9a8e-4ab6-830b-6052efbdb5e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:47.899181 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-acd59ec8-9a8e-4ab6-830b-6052efbdb5e8 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:47.900421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-acd59ec8-9a8e-4ab6-830b-6052efbdb5e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:47.923315 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:47.966233 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71cafdd-e90a-47b0-9ba6-4a77e0e34812 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:47.999502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 276/564] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:48.000370 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd59ec8-9a8e-4ab6-830b-6052efbdb5e8 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:48.012587 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59383380-fee3-4cb8-b087-18c3d42e4dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:48.013718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59383380-fee3-4cb8-b087-18c3d42e4dbf None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:48.014190 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59383380-fee3-4cb8-b087-18c3d42e4dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:48.014750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59383380-fee3-4cb8-b087-18c3d42e4dbf None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:48.027594 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59383380-fee3-4cb8-b087-18c3d42e4dbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:48.029061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59383380-fee3-4cb8-b087-18c3d42e4dbf None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:48.048152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-59383380-fee3-4cb8-b087-18c3d42e4dbf None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:48.048152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:48.048152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:48.048152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:48.048152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:48.048152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:48.048152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:48.048152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:48.048152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:48.048152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:48.048152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:48.048152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:48.048152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:48.048152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:48.048152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:48.049465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:48.049465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:48.049465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:48.049465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:48.049465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:48.049465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:48.049465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:48.049465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:48.049465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:48.049465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:48.049465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:48.049465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:48.049465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:48.049465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:48.049465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:48.049465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:48.049465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:48.050782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:48.050782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:48.050782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:48.050782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:48.050782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 289/565] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:48.051179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a7807-d4ba-43a7-b9f9-3c54545f4349 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:48.051383 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a7807-d4ba-43a7-b9f9-3c54545f4349 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:48.051651 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a39a7807-d4ba-43a7-b9f9-3c54545f4349 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:48.054461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 290/566] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:48.055784 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a7807-d4ba-43a7-b9f9-3c54545f4349 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:48.121288 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:48.125094 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acd59ec8-9a8e-4ab6-830b-6052efbdb5e8 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:48.161251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b72a5-5d9e-49cc-901c-41ad8c256164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:48.161925 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b72a5-5d9e-49cc-901c-41ad8c256164 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:48.162205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b72a5-5d9e-49cc-901c-41ad8c256164 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:48.162513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-436b72a5-5d9e-49cc-901c-41ad8c256164 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:48.163271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b72a5-5d9e-49cc-901c-41ad8c256164 None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:48.163578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b72a5-5d9e-49cc-901c-41ad8c256164 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:48.164965 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-436b72a5-5d9e-49cc-901c-41ad8c256164 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:48.165257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a39a7807-d4ba-43a7-b9f9-3c54545f4349 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:48.174049 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 277/567] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:53:47 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:48.174911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b72a5-5d9e-49cc-901c-41ad8c256164 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:48.185989 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:48.207157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2727f56-db7a-4e0f-8d1f-3dae08de7e84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:48.207845 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2727f56-db7a-4e0f-8d1f-3dae08de7e84 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:48.208132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2727f56-db7a-4e0f-8d1f-3dae08de7e84 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:48.208861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2727f56-db7a-4e0f-8d1f-3dae08de7e84 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:48.209146 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2727f56-db7a-4e0f-8d1f-3dae08de7e84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:48.209459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2727f56-db7a-4e0f-8d1f-3dae08de7e84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:48.212006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2727f56-db7a-4e0f-8d1f-3dae08de7e84 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:48.223945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 291/568] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:48.225425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2727f56-db7a-4e0f-8d1f-3dae08de7e84 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:48.284637 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-436b72a5-5d9e-49cc-901c-41ad8c256164 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:48.319850 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf1a9297-7449-4d18-9a8f-5ef9b247a980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:48.320662 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a9297-7449-4d18-9a8f-5ef9b247a980 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:48.320948 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a9297-7449-4d18-9a8f-5ef9b247a980 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:48.321238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf1a9297-7449-4d18-9a8f-5ef9b247a980 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:48.330020 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2727f56-db7a-4e0f-8d1f-3dae08de7e84 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:48.332669 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf1a9297-7449-4d18-9a8f-5ef9b247a980 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:48.340491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-bf1a9297-7449-4d18-9a8f-5ef9b247a980 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:48.340491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:48.340491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:48.340491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:48.340491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:48.340491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:48.340491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:48.340491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:48.340491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:48.340491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:48.340491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:48.340491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:48.340491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:48.340491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:48.340491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:48.340491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:48.341508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:48.341508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:48.341508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:48.341508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:48.341508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:48.341508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:48.341508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:48.341508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:48.341508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:48.341508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:48.341508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:48.341508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:48.341508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:48.341508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:48.341508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:48.341508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:48.342356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:48.342356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:48.342356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:48.342356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:48.342356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:48.342356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:48.342356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:48.342356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:48.342356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:48.342356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:48.342356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:48.342356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:48.342356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 278/569] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:53:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:48.343666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012e234-06cf-4bba-8ecd-96c18a30cffb None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:48.344250 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012e234-06cf-4bba-8ecd-96c18a30cffb None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:48.344250 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c012e234-06cf-4bba-8ecd-96c18a30cffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:48.361371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c012e234-06cf-4bba-8ecd-96c18a30cffb None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:48.362627 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c012e234-06cf-4bba-8ecd-96c18a30cffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:48.367978 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa03f0-47ef-410a-8d4e-bc446ba87690 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:48.368656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa03f0-47ef-410a-8d4e-bc446ba87690 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:48.368938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa03f0-47ef-410a-8d4e-bc446ba87690 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:48.369218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cafa03f0-47ef-410a-8d4e-bc446ba87690 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:48.369958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa03f0-47ef-410a-8d4e-bc446ba87690 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:48.370270 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa03f0-47ef-410a-8d4e-bc446ba87690 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:48.371724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cafa03f0-47ef-410a-8d4e-bc446ba87690 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:48.380676 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:48.385351 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 292/570] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:48.387047 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa03f0-47ef-410a-8d4e-bc446ba87690 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:48.435268 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:48.442059 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 279/571] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:48.442858 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012e234-06cf-4bba-8ecd-96c18a30cffb None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:48.507152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cafa03f0-47ef-410a-8d4e-bc446ba87690 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:48.542070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cdbd3-1fa0-488a-8a3b-0c32ab91ef75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:48.542891 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cdbd3-1fa0-488a-8a3b-0c32ab91ef75 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:48.543175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cdbd3-1fa0-488a-8a3b-0c32ab91ef75 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:48.543460 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a25cdbd3-1fa0-488a-8a3b-0c32ab91ef75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:48.554076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25cdbd3-1fa0-488a-8a3b-0c32ab91ef75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:48.555810 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a25cdbd3-1fa0-488a-8a3b-0c32ab91ef75 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:48.556373 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c012e234-06cf-4bba-8ecd-96c18a30cffb None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:48.574218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a25cdbd3-1fa0-488a-8a3b-0c32ab91ef75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:48.574218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:48.574218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:48.574218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:48.574218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:48.574218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:48.574218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:48.574218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:48.574218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:48.574218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:48.574218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:48.574218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:48.574218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:48.574218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:48.574218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:48.575425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:48.575425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:48.575425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:48.575425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:48.575425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:48.575425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:48.575425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:48.575425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:48.575425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:48.575425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:48.575425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:48.575425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:48.575425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:48.575425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:48.575425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:48.575425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:48.575425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:48.576535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:48.576535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:48.576535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:48.576535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:48.576535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 293/572] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:48.576535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cdbd3-1fa0-488a-8a3b-0c32ab91ef75 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:48.593815 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179faf0-d50d-4586-a537-0247f7f41dfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:48.594651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179faf0-d50d-4586-a537-0247f7f41dfe None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:48.594935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179faf0-d50d-4586-a537-0247f7f41dfe None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:48.595208 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8179faf0-d50d-4586-a537-0247f7f41dfe None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:48.602128 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179faf0-d50d-4586-a537-0247f7f41dfe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:48.603632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8179faf0-d50d-4586-a537-0247f7f41dfe None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:48.617859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-8179faf0-d50d-4586-a537-0247f7f41dfe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:48.617859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:48.617859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:48.617859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:48.617859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:48.617859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:48.617859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:48.617859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:48.617859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:48.617859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:48.617859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:48.617859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:48.617859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:53:48.617859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:53:48.617859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:53:48.618878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:53:48.618878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:48.618878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:48.618878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:48.618878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:48.618878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:48.618878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:48.618878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:48.618878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:48.618878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:48.618878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:48.618878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:48.618878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:48.618878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:48.618878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:48.618878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:48.618878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:48.619751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:53:48.619751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:53:48.619751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:53:48.619751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:53:48.619751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:53:48.619751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:53:48.619751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:48.619751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:48.622726 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 280/573] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:48.624309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179faf0-d50d-4586-a537-0247f7f41dfe None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:48.640453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:48.676287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:48.691995 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a25cdbd3-1fa0-488a-8a3b-0c32ab91ef75 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:48.732012 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8179faf0-d50d-4586-a537-0247f7f41dfe None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:48.738358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bcd975-f880-47a6-943c-98268d54b4d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:48.739389 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcd975-f880-47a6-943c-98268d54b4d9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:48.739819 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcd975-f880-47a6-943c-98268d54b4d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:48.740242 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8bcd975-f880-47a6-943c-98268d54b4d9 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:48.741413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bcd975-f880-47a6-943c-98268d54b4d9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:48.741897 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bcd975-f880-47a6-943c-98268d54b4d9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:48.744107 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8bcd975-f880-47a6-943c-98268d54b4d9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:48.756972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 294/574] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:53:48 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:48.758205 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bcd975-f880-47a6-943c-98268d54b4d9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:48.769851 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41ab6a-f686-4dcd-b1cf-d8ab3f7cdd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:48.770691 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41ab6a-f686-4dcd-b1cf-d8ab3f7cdd60 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:48.770977 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41ab6a-f686-4dcd-b1cf-d8ab3f7cdd60 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:48.771256 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b41ab6a-f686-4dcd-b1cf-d8ab3f7cdd60 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:48.778682 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41ab6a-f686-4dcd-b1cf-d8ab3f7cdd60 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:48.780157 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b41ab6a-f686-4dcd-b1cf-d8ab3f7cdd60 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:48.794809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-4b41ab6a-f686-4dcd-b1cf-d8ab3f7cdd60 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:48.794809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:48.794809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:48.794809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:48.794809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:48.794809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:48.794809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:48.794809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:48.794809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:48.794809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:48.794809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:48.794809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:48.794809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:53:48.794809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:53:48.794809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:53:48.795894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:53:48.795894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:48.795894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:48.795894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:48.795894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:48.795894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:48.795894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:48.795894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:48.795894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:48.795894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:48.795894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:48.795894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:48.795894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:48.795894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:48.795894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:48.795894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:48.795894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:48.796769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:53:48.796769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:53:48.796769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:53:48.796769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:53:48.796769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:53:48.796769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:53:48.796769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:48.796769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:48.796769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 281/575] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:48 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:48.797777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41ab6a-f686-4dcd-b1cf-d8ab3f7cdd60 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:48.872167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:48.892290 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8bcd975-f880-47a6-943c-98268d54b4d9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:48.905154 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b41ab6a-f686-4dcd-b1cf-d8ab3f7cdd60 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:48.928130 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575f4ce-e69b-4b8c-bc6a-fe352acff2a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:48.928942 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575f4ce-e69b-4b8c-bc6a-fe352acff2a3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:48.929287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575f4ce-e69b-4b8c-bc6a-fe352acff2a3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:48.929575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a575f4ce-e69b-4b8c-bc6a-fe352acff2a3 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:48.944274 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a575f4ce-e69b-4b8c-bc6a-fe352acff2a3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:48.949309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5319ad-3f2e-435f-ad44-f0148f1ec63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:48.949979 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5319ad-3f2e-435f-ad44-f0148f1ec63c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:48.950267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5319ad-3f2e-435f-ad44-f0148f1ec63c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:48.950579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5319ad-3f2e-435f-ad44-f0148f1ec63c None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:48.951333 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5319ad-3f2e-435f-ad44-f0148f1ec63c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:48.951648 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5319ad-3f2e-435f-ad44-f0148f1ec63c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:48.952939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a575f4ce-e69b-4b8c-bc6a-fe352acff2a3 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:48.952939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:48.952939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:48.952939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:48.952939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:48.952939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:48.952939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:48.952939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:48.952939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:48.952939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:48.952939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:48.952939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:48.952939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:48.952939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:48.952939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:48.952939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:48.954046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:48.954046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:48.954046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:48.954046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:48.954046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:48.954046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:48.954046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:48.954046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:48.954046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:48.954046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:48.954046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:48.954046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:48.954046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:48.954046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:48.954046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:48.954046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:48.960135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:48.960135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:48.960135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:48.960135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:48.960135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:48.960135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:48.960135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:48.960135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:48.960135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:48.960135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:48.960135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:48.960135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:48.960776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5319ad-3f2e-435f-ad44-f0148f1ec63c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:48.960852 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 295/576] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:53:48 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:48.960852 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575f4ce-e69b-4b8c-bc6a-fe352acff2a3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:48.971340 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 282/577] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:48.972224 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5319ad-3f2e-435f-ad44-f0148f1ec63c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:48.974342 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:49.083224 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a575f4ce-e69b-4b8c-bc6a-fe352acff2a3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:49.100287 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5319ad-3f2e-435f-ad44-f0148f1ec63c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:49.122068 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24554dc-ef18-45e7-a7f1-ec451808ec54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:49.123191 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24554dc-ef18-45e7-a7f1-ec451808ec54 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:49.123403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24554dc-ef18-45e7-a7f1-ec451808ec54 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:49.123683 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24554dc-ef18-45e7-a7f1-ec451808ec54 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:49.125087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24554dc-ef18-45e7-a7f1-ec451808ec54 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:49.125401 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24554dc-ef18-45e7-a7f1-ec451808ec54 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:49.127292 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24554dc-ef18-45e7-a7f1-ec451808ec54 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:49.137255 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:49.140646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 296/578] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:53:48 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:49.148365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24554dc-ef18-45e7-a7f1-ec451808ec54 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:49.150003 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971f9a2-5e80-404e-90d0-4498090f9a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:49.150862 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971f9a2-5e80-404e-90d0-4498090f9a8e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:49.151145 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971f9a2-5e80-404e-90d0-4498090f9a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:49.151419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b971f9a2-5e80-404e-90d0-4498090f9a8e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:49.164222 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971f9a2-5e80-404e-90d0-4498090f9a8e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:49.165716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b971f9a2-5e80-404e-90d0-4498090f9a8e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:49.184251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-b971f9a2-5e80-404e-90d0-4498090f9a8e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:49.184251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:49.184251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:49.184251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:49.184251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:49.184251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:49.184251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:49.184251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:49.184251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:49.184251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:49.184251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:49.184251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:49.184251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:49.184251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:49.184251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:49.185362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:49.185362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:49.185362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:49.185362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:49.185362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:49.185362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:49.185362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:49.185362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:49.185362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:49.185362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:49.185362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:49.185362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:49.185362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:49.185362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:49.185362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:49.185362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:49.185362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:49.187558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:49.187558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:49.187558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:49.187558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:49.187558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 283/579] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:49.187558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971f9a2-5e80-404e-90d0-4498090f9a8e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:49.220530 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:49.253880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24554dc-ef18-45e7-a7f1-ec451808ec54 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:49.288901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589fcc27-85b3-4828-b6fb-f3fe8fa7dfb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:49.289509 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b971f9a2-5e80-404e-90d0-4498090f9a8e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:49.289625 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589fcc27-85b3-4828-b6fb-f3fe8fa7dfb4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:49.289846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589fcc27-85b3-4828-b6fb-f3fe8fa7dfb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:49.290118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-589fcc27-85b3-4828-b6fb-f3fe8fa7dfb4 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:49.290976 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589fcc27-85b3-4828-b6fb-f3fe8fa7dfb4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:49.291287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589fcc27-85b3-4828-b6fb-f3fe8fa7dfb4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:49.292746 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-589fcc27-85b3-4828-b6fb-f3fe8fa7dfb4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:49.310743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 297/580] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:53:49 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:49.310743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589fcc27-85b3-4828-b6fb-f3fe8fa7dfb4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:49.326285 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931b795-7a0a-4bde-829e-6320c6166017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:49.327067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931b795-7a0a-4bde-829e-6320c6166017 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:49.327360 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931b795-7a0a-4bde-829e-6320c6166017 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:49.327631 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a931b795-7a0a-4bde-829e-6320c6166017 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:49.328410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931b795-7a0a-4bde-829e-6320c6166017 None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:49.328724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931b795-7a0a-4bde-829e-6320c6166017 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:49.330138 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a931b795-7a0a-4bde-829e-6320c6166017 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:49.339010 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 284/581] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:53:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:49.339863 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931b795-7a0a-4bde-829e-6320c6166017 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:49.390067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:49.439320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-589fcc27-85b3-4828-b6fb-f3fe8fa7dfb4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:49.451258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a931b795-7a0a-4bde-829e-6320c6166017 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:49.469306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:49.476249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035eeb0-3ac6-4f55-8a82-a247e83ff0b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:49.477066 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035eeb0-3ac6-4f55-8a82-a247e83ff0b0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:49.477346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035eeb0-3ac6-4f55-8a82-a247e83ff0b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:49.477622 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b035eeb0-3ac6-4f55-8a82-a247e83ff0b0 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:49.487501 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035eeb0-3ac6-4f55-8a82-a247e83ff0b0 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:49.488074 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55cea5-e784-40f6-8e14-476b4995f160 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:49.488964 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b035eeb0-3ac6-4f55-8a82-a247e83ff0b0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:49.489123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55cea5-e784-40f6-8e14-476b4995f160 None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:49.489470 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55cea5-e784-40f6-8e14-476b4995f160 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:49.489810 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e55cea5-e784-40f6-8e14-476b4995f160 None admin] PATH_INFO: `/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/ef5dac3f311e43809a46995bab56e780/roles/1f7cf26664d04e5e9331ead293476442` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:49.504113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b035eeb0-3ac6-4f55-8a82-a247e83ff0b0 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 13:53:49.504113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:49.504113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:49.504113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:49.504113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:49.504113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:49.504113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:49.504113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:49.504113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:49.504113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:49.504113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:49.504113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:49.504113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:53:49.504113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:53:49.504113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:53:49.504113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:53:49.505261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:49.505261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:49.505261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:53:49.505261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:49.505261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:53:49.505261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:49.505261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:49.505261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:49.505261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:49.505261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:49.505261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:49.505261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:49.505261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:49.505261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:49.505261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:49.505261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:49.505261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:49.505261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:49.506204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:49.506204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:49.506204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:53:49.506204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:53:49.506204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:53:49.506204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:53:49.506204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:53:49.506204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:53:49.506204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 13:53:49.506204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:49.506204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 298/582] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:49 2022] GET /identity/v3/users/cinder => generated 84 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:49.506204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035eeb0-3ac6-4f55-8a82-a247e83ff0b0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:49.507299 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55cea5-e784-40f6-8e14-476b4995f160 None admin] RBAC: Authorizing `identity:create_grant(project_id=08ee65f8b338491f80e7beac5e316b23, user_id=ef5dac3f311e43809a46995bab56e780, role_id=1f7cf26664d04e5e9331ead293476442)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:49.508827 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e55cea5-e784-40f6-8e14-476b4995f160 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:49.522457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 285/583] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:53:49 2022] PUT /identity/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/ef5dac3f311e43809a46995bab56e780/roles/1f7cf26664d04e5e9331ead293476442 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:53:49.523309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55cea5-e784-40f6-8e14-476b4995f160 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:49.627432 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b035eeb0-3ac6-4f55-8a82-a247e83ff0b0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:49.641273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:49.649961 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e55cea5-e784-40f6-8e14-476b4995f160 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:49.668755 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e572c-03d2-4ae3-ae62-56cbffc2a1c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:49.669446 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e572c-03d2-4ae3-ae62-56cbffc2a1c1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:49.669736 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e572c-03d2-4ae3-ae62-56cbffc2a1c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:49.670017 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127e572c-03d2-4ae3-ae62-56cbffc2a1c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:49.671513 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e572c-03d2-4ae3-ae62-56cbffc2a1c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:49.671831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e572c-03d2-4ae3-ae62-56cbffc2a1c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:49.673250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-127e572c-03d2-4ae3-ae62-56cbffc2a1c1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:49.690117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 299/584] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:49.690545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e572c-03d2-4ae3-ae62-56cbffc2a1c1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:49.702713 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f73c66-5d65-4172-8a80-4301b9e1e088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:49.703489 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f73c66-5d65-4172-8a80-4301b9e1e088 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:49.703833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f73c66-5d65-4172-8a80-4301b9e1e088 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:49.704122 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64f73c66-5d65-4172-8a80-4301b9e1e088 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:49.704983 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f73c66-5d65-4172-8a80-4301b9e1e088 None admin] RBAC: Adding query filter params (role.id=1f7cf26664d04e5e9331ead293476442, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=d9c1369033a44c1c81d4c1c9ff2bbedb) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:49.705303 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f73c66-5d65-4172-8a80-4301b9e1e088 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:49.707020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64f73c66-5d65-4172-8a80-4301b9e1e088 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:49.713283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 286/585] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:53:49 2022] GET /identity/v3/role_assignments?user.id=d9c1369033a44c1c81d4c1c9ff2bbedb&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=1f7cf26664d04e5e9331ead293476442 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:49.714140 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f73c66-5d65-4172-8a80-4301b9e1e088 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:49.768252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:49.838354 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-127e572c-03d2-4ae3-ae62-56cbffc2a1c1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:49.860987 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64f73c66-5d65-4172-8a80-4301b9e1e088 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:49.881667 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539f622-74eb-4915-89ec-cd3bed6004ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:49.882628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539f622-74eb-4915-89ec-cd3bed6004ce None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:49.882991 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539f622-74eb-4915-89ec-cd3bed6004ce None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:49.883342 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5539f622-74eb-4915-89ec-cd3bed6004ce None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:49.897469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539f622-74eb-4915-89ec-cd3bed6004ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:49.899180 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5539f622-74eb-4915-89ec-cd3bed6004ce None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:49.919360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-5539f622-74eb-4915-89ec-cd3bed6004ce None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:49.919360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:49.919360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:49.919360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:49.919360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:49.919360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:49.919360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:49.919360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:49.919360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:49.919360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:49.919360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:49.919360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:49.919360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:49.919360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:49.919360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:49.920339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:49.920339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:49.920339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:49.920339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:49.920339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:49.920339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:49.920339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:49.920339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:49.920339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:49.920339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:49.920339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:49.920339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:49.920339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:49.920339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:49.920339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:49.920339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:49.920339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:49.921270 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:49.921270 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:49.921270 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:49.921270 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:49.921270 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 300/586] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:49.921553 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539f622-74eb-4915-89ec-cd3bed6004ce None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:49.923912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f0f09-fceb-40cd-be5e-fbed8e2ec8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:49.924684 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f0f09-fceb-40cd-be5e-fbed8e2ec8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:49.924972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f0f09-fceb-40cd-be5e-fbed8e2ec8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:49.925254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006f0f09-fceb-40cd-be5e-fbed8e2ec8e2 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:49.926436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f0f09-fceb-40cd-be5e-fbed8e2ec8e2 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:49.926608 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f0f09-fceb-40cd-be5e-fbed8e2ec8e2 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:49.931589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006f0f09-fceb-40cd-be5e-fbed8e2ec8e2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:49.955862 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 287/587] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Jul 11 13:53:49 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:49.956625 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f0f09-fceb-40cd-be5e-fbed8e2ec8e2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:49.982163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:50.086348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5539f622-74eb-4915-89ec-cd3bed6004ce None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:50.086348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:50.105901 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006f0f09-fceb-40cd-be5e-fbed8e2ec8e2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:50.132361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed4ed2-2fc9-4d84-b364-6d61d7683f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:50.133183 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed4ed2-2fc9-4d84-b364-6d61d7683f83 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:50.133778 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed4ed2-2fc9-4d84-b364-6d61d7683f83 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:50.134070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11ed4ed2-2fc9-4d84-b364-6d61d7683f83 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:50.144421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed4ed2-2fc9-4d84-b364-6d61d7683f83 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:50.145995 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11ed4ed2-2fc9-4d84-b364-6d61d7683f83 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:50.163471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-11ed4ed2-2fc9-4d84-b364-6d61d7683f83 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 11 13:53:50.163471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:50.163471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:50.163471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:50.163471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:50.163471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:50.163471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:50.163471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:50.163471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:50.163471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:50.163471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:50.163471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:50.163471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:53:50.163471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:53:50.163471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:53:50.163471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:53:50.164459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:50.164459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:50.164459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:53:50.164459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:50.164459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:53:50.164459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:50.164459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:50.164459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:50.164459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:50.164459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:50.164459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:50.164459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:50.164459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:50.164459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:50.164459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:50.164459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:50.164459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:50.164459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:50.165805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:50.165805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:50.165805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:53:50.165805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:53:50.165805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:53:50.165805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:53:50.165805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:53:50.165805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:53:50.165805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 11 13:53:50.165805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:50.165805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 301/588] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:49 2022] GET /identity/v3/users/glance => generated 84 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:50.165805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed4ed2-2fc9-4d84-b364-6d61d7683f83 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:50.172783 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff4432-cc76-45ea-a1aa-f8c7828e4b77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:50.173535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff4432-cc76-45ea-a1aa-f8c7828e4b77 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:50.173820 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff4432-cc76-45ea-a1aa-f8c7828e4b77 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:50.174099 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ff4432-cc76-45ea-a1aa-f8c7828e4b77 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:50.181122 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff4432-cc76-45ea-a1aa-f8c7828e4b77 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:50.181722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff4432-cc76-45ea-a1aa-f8c7828e4b77 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:50.186858 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ff4432-cc76-45ea-a1aa-f8c7828e4b77 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:50.220539 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 288/589] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:50.221597 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff4432-cc76-45ea-a1aa-f8c7828e4b77 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:50.342636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11ed4ed2-2fc9-4d84-b364-6d61d7683f83 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:50.350731 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:50.363362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ff4432-cc76-45ea-a1aa-f8c7828e4b77 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:50.414253 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db256950-699d-4f70-a80d-f92ef407d676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:50.416107 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db256950-699d-4f70-a80d-f92ef407d676 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:50.416625 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db256950-699d-4f70-a80d-f92ef407d676 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:50.417136 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db256950-699d-4f70-a80d-f92ef407d676 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:50.420438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7dd861-8b95-41bf-b2bf-6cb8f22e0b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:50.421449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7dd861-8b95-41bf-b2bf-6cb8f22e0b9a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:50.421918 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7dd861-8b95-41bf-b2bf-6cb8f22e0b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:50.422411 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7dd861-8b95-41bf-b2bf-6cb8f22e0b9a None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:50.423502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7dd861-8b95-41bf-b2bf-6cb8f22e0b9a None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:50.423999 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7dd861-8b95-41bf-b2bf-6cb8f22e0b9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:50.425653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7dd861-8b95-41bf-b2bf-6cb8f22e0b9a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:50.443679 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db256950-699d-4f70-a80d-f92ef407d676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:50.446138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db256950-699d-4f70-a80d-f92ef407d676 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:50.448918 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 289/590] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Jul 11 13:53:50 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:50.450655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7dd861-8b95-41bf-b2bf-6cb8f22e0b9a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:50.472838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-db256950-699d-4f70-a80d-f92ef407d676 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:50.472838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:50.472838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:50.472838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:50.472838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:50.472838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:50.472838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:50.472838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:50.472838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:50.472838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:50.472838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:50.472838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:50.472838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:50.472838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:50.472838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:50.473760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:50.473760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:50.473760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:50.473760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:50.473760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:50.473760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:50.473760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:50.473760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:50.473760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:50.473760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:50.473760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:50.473760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:50.473760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:50.473760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:50.473760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:50.473760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:50.473760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:50.475803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:50.475803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:50.475803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:50.475803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:50.475803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 302/591] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 309 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:50.475803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db256950-699d-4f70-a80d-f92ef407d676 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:50.495369 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:50.608709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db256950-699d-4f70-a80d-f92ef407d676 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:50.652472 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562fed9-c807-4d94-8a28-b9c363b9f87c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:50.653684 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562fed9-c807-4d94-8a28-b9c363b9f87c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:50.653981 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562fed9-c807-4d94-8a28-b9c363b9f87c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:50.654259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a562fed9-c807-4d94-8a28-b9c363b9f87c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:50.659447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7dd861-8b95-41bf-b2bf-6cb8f22e0b9a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:50.668841 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a562fed9-c807-4d94-8a28-b9c363b9f87c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:50.670399 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a562fed9-c807-4d94-8a28-b9c363b9f87c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:50.696590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a562fed9-c807-4d94-8a28-b9c363b9f87c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:50.696590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:50.696590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:50.696590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:50.696590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:50.696590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:50.696590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:50.696590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:50.696590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:50.696590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:50.696590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:50.696590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:50.696590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:50.696590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:50.696590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:50.698603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:50.698603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:50.698603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:50.698603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:50.698603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:50.698603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:50.698603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:50.698603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:50.698603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:50.698603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:50.698603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:50.698603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:50.698603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:50.698603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:50.698603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:50.698603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:50.698603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:50.700950 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:50.700950 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:50.700950 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:50.700950 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:50.700950 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 303/592] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:50.700950 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562fed9-c807-4d94-8a28-b9c363b9f87c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:50.739531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94338dfc-7b62-4fd5-aabe-abf626712b67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:50.742371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94338dfc-7b62-4fd5-aabe-abf626712b67 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:50.742894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94338dfc-7b62-4fd5-aabe-abf626712b67 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:50.743585 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94338dfc-7b62-4fd5-aabe-abf626712b67 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:50.749327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:50.762714 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94338dfc-7b62-4fd5-aabe-abf626712b67 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:50.767785 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94338dfc-7b62-4fd5-aabe-abf626712b67 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:50.794296 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:50.798830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-94338dfc-7b62-4fd5-aabe-abf626712b67 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 13:53:50.798830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:50.798830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:50.798830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:50.798830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:50.798830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:50.798830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:50.798830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:50.798830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:50.798830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:50.798830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:50.798830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:50.798830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:53:50.798830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:53:50.798830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:53:50.798830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:53:50.800188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:50.800188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:50.800188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:53:50.800188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:50.800188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:53:50.800188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:50.800188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:50.800188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:50.800188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:50.800188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:50.800188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:50.800188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:50.800188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:50.800188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:50.800188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:50.800188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:50.800188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:50.800188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:50.801514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:50.801514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:50.801514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:53:50.801514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:53:50.801514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:53:50.801514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:53:50.801514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:53:50.801514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:53:50.801514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 13:53:50.801514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:50.804479 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 290/593] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:53:50 2022] GET /identity/v3/users/neutron => generated 85 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:50.805925 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94338dfc-7b62-4fd5-aabe-abf626712b67 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:50.848078 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a562fed9-c807-4d94-8a28-b9c363b9f87c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:50.892716 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca036db-7657-4e64-8717-a079a0405636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:50.893643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca036db-7657-4e64-8717-a079a0405636 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:50.893963 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca036db-7657-4e64-8717-a079a0405636 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:50.894383 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca036db-7657-4e64-8717-a079a0405636 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:50.895301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca036db-7657-4e64-8717-a079a0405636 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:50.895667 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca036db-7657-4e64-8717-a079a0405636 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:50.897150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca036db-7657-4e64-8717-a079a0405636 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:50.911295 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 304/594] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:50.912364 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca036db-7657-4e64-8717-a079a0405636 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:50.950675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94338dfc-7b62-4fd5-aabe-abf626712b67 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:50.991242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a98845-39a5-4696-bd94-8dc2345b55af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:50.992742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a98845-39a5-4696-bd94-8dc2345b55af None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:50.993662 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a98845-39a5-4696-bd94-8dc2345b55af None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:50.994465 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a98845-39a5-4696-bd94-8dc2345b55af None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:50.995757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a98845-39a5-4696-bd94-8dc2345b55af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:50.996292 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a98845-39a5-4696-bd94-8dc2345b55af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:50.998551 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21a98845-39a5-4696-bd94-8dc2345b55af None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:51.020148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 291/595] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:51.021426 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a98845-39a5-4696-bd94-8dc2345b55af None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:51.021670 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca036db-7657-4e64-8717-a079a0405636 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:51.054371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:51.066027 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea1e22-c50d-4c80-8540-f82c4a2af685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:51.067057 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea1e22-c50d-4c80-8540-f82c4a2af685 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:51.067364 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea1e22-c50d-4c80-8540-f82c4a2af685 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:51.067742 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fea1e22-c50d-4c80-8540-f82c4a2af685 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:51.069018 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea1e22-c50d-4c80-8540-f82c4a2af685 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:51.069208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea1e22-c50d-4c80-8540-f82c4a2af685 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:51.071265 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fea1e22-c50d-4c80-8540-f82c4a2af685 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:51.098761 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 305/596] 10.222.0.49 () {56 vars in 1139 bytes} [Mon Jul 11 13:53:50 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:51.099954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea1e22-c50d-4c80-8540-f82c4a2af685 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:51.102527 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:51.153983 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a98845-39a5-4696-bd94-8dc2345b55af None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:51.193651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f89f4-fcf4-42d6-9e39-db6dbf220a47 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:51.194496 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f89f4-fcf4-42d6-9e39-db6dbf220a47 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:51.194780 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f89f4-fcf4-42d6-9e39-db6dbf220a47 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:51.195051 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f89f4-fcf4-42d6-9e39-db6dbf220a47 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:51.206181 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f89f4-fcf4-42d6-9e39-db6dbf220a47 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:51.207772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f89f4-fcf4-42d6-9e39-db6dbf220a47 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:51.219185 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fea1e22-c50d-4c80-8540-f82c4a2af685 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:51.223670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-451f89f4-fcf4-42d6-9e39-db6dbf220a47 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:51.223670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:51.223670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:51.223670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:51.223670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:51.223670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:51.223670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:51.223670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:51.223670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:51.223670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:51.223670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:51.223670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:51.223670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:53:51.223670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:53:51.223670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:53:51.224626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:53:51.224626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:51.224626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:51.224626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:51.224626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:51.224626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:51.224626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:51.224626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:51.224626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:51.224626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:51.224626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:51.224626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:51.224626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:51.224626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:51.224626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:51.224626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:51.224626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:51.225506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:53:51.225506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:53:51.225506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:53:51.225506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:53:51.225506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:53:51.225506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:53:51.225506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:51.225506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:51.225506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 292/597] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:51.225999 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f89f4-fcf4-42d6-9e39-db6dbf220a47 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:51.263338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3d8e7-551a-448b-aea4-389fa7ad9d91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:51.265103 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3d8e7-551a-448b-aea4-389fa7ad9d91 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:51.265609 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3d8e7-551a-448b-aea4-389fa7ad9d91 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:51.266130 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a3d8e7-551a-448b-aea4-389fa7ad9d91 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:51.274675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3d8e7-551a-448b-aea4-389fa7ad9d91 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:51.276627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35a3d8e7-551a-448b-aea4-389fa7ad9d91 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:51.292972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-35a3d8e7-551a-448b-aea4-389fa7ad9d91 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:51.292972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:51.292972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:51.292972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:51.292972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:51.292972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:51.292972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:51.292972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:51.292972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:51.292972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:51.292972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:51.292972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:51.292972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:53:51.292972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:53:51.292972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:53:51.293927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:53:51.293927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:51.293927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:51.293927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:51.293927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:51.293927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:51.293927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:51.293927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:51.293927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:51.293927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:51.293927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:51.293927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:51.293927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:51.293927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:51.293927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:51.293927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:51.293927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:51.294832 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:53:51.294832 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:53:51.294832 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:53:51.294832 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:53:51.294832 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:53:51.294832 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:53:51.294832 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:51.294832 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:51.298094 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 306/598] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:51.299341 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3d8e7-551a-448b-aea4-389fa7ad9d91 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:51.349413 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f89f4-fcf4-42d6-9e39-db6dbf220a47 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:51.352791 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:51.371192 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:51.400448 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794404ec-4ba3-45b8-94eb-3cd2994ce9da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:51.402403 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794404ec-4ba3-45b8-94eb-3cd2994ce9da None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:51.402891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794404ec-4ba3-45b8-94eb-3cd2994ce9da None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:51.403377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794404ec-4ba3-45b8-94eb-3cd2994ce9da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:51.421472 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794404ec-4ba3-45b8-94eb-3cd2994ce9da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:51.422630 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35a3d8e7-551a-448b-aea4-389fa7ad9d91 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:51.423675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794404ec-4ba3-45b8-94eb-3cd2994ce9da None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:51.454230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-794404ec-4ba3-45b8-94eb-3cd2994ce9da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:51.454230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:51.454230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:51.454230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:51.454230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:51.454230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:51.454230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:51.454230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:51.454230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:51.454230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:51.454230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:51.454230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:51.454230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:51.454230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:51.454230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:51.455190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:51.455190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:51.455190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:51.455190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:51.455190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:51.455190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:51.455190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:51.455190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:51.455190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:51.455190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:51.455190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:51.455190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:51.455190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:51.455190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:51.455190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:51.455190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:51.455190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:51.456060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:51.456060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:51.456060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:51.456060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:51.458846 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 293/599] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:51.459910 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794404ec-4ba3-45b8-94eb-3cd2994ce9da None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:51.475731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45f874-168b-4eac-bc74-69fa676faafc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:51.476961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45f874-168b-4eac-bc74-69fa676faafc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:51.477527 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45f874-168b-4eac-bc74-69fa676faafc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:51.477921 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc45f874-168b-4eac-bc74-69fa676faafc None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:51.479212 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45f874-168b-4eac-bc74-69fa676faafc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:51.479621 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45f874-168b-4eac-bc74-69fa676faafc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:51.481667 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc45f874-168b-4eac-bc74-69fa676faafc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:51.498713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 307/600] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:53:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:51.501816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45f874-168b-4eac-bc74-69fa676faafc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:51.622121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794404ec-4ba3-45b8-94eb-3cd2994ce9da None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:51.646392 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc45f874-168b-4eac-bc74-69fa676faafc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:51.651250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:51.669727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39949375-0325-48f1-b2a7-1cab915e60c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:51.671581 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39949375-0325-48f1-b2a7-1cab915e60c9 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:51.672681 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39949375-0325-48f1-b2a7-1cab915e60c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:51.673153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39949375-0325-48f1-b2a7-1cab915e60c9 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:51.674479 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39949375-0325-48f1-b2a7-1cab915e60c9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:51.674973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39949375-0325-48f1-b2a7-1cab915e60c9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:51.676677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39949375-0325-48f1-b2a7-1cab915e60c9 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:51.699485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 294/601] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:53:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:51.700703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39949375-0325-48f1-b2a7-1cab915e60c9 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:51.705790 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee0ea2-c4cf-4c7e-9640-a465d3366cc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:51.706398 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee0ea2-c4cf-4c7e-9640-a465d3366cc8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:51.706734 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee0ea2-c4cf-4c7e-9640-a465d3366cc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:51.707081 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ee0ea2-c4cf-4c7e-9640-a465d3366cc8 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:51.708010 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee0ea2-c4cf-4c7e-9640-a465d3366cc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:51.708341 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee0ea2-c4cf-4c7e-9640-a465d3366cc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:51.710139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ee0ea2-c4cf-4c7e-9640-a465d3366cc8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:51.730420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 308/602] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:51.730420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee0ea2-c4cf-4c7e-9640-a465d3366cc8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:51.742909 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:51.826623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39949375-0325-48f1-b2a7-1cab915e60c9 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:51.869998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e21a98-1bd9-477c-b277-ba8c3eed896f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:51.870968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e21a98-1bd9-477c-b277-ba8c3eed896f None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:51.871256 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e21a98-1bd9-477c-b277-ba8c3eed896f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:51.871958 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e21a98-1bd9-477c-b277-ba8c3eed896f None admin] PATH_INFO: `/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/6bb1616f632d4c67828cce3fc41945b7/roles/1f7cf26664d04e5e9331ead293476442` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:51.874986 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ee0ea2-c4cf-4c7e-9640-a465d3366cc8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:51.898863 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e21a98-1bd9-477c-b277-ba8c3eed896f None admin] RBAC: Authorizing `identity:create_grant(project_id=08ee65f8b338491f80e7beac5e316b23, user_id=6bb1616f632d4c67828cce3fc41945b7, role_id=1f7cf26664d04e5e9331ead293476442)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:51.900351 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e21a98-1bd9-477c-b277-ba8c3eed896f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:51.920002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 295/603] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:53:51 2022] PUT /identity/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/6bb1616f632d4c67828cce3fc41945b7/roles/1f7cf26664d04e5e9331ead293476442 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:53:51.921875 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e21a98-1bd9-477c-b277-ba8c3eed896f None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:51.933215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16973409-6295-4da7-b673-5ebfa85c852d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:51.934914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16973409-6295-4da7-b673-5ebfa85c852d None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:51.935419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16973409-6295-4da7-b673-5ebfa85c852d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:51.935915 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16973409-6295-4da7-b673-5ebfa85c852d None admin] PATH_INFO: `/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/f80d59fec4994e689e857ba2087d3a23/roles/1f7cf26664d04e5e9331ead293476442` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:51.963992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16973409-6295-4da7-b673-5ebfa85c852d None admin] RBAC: Authorizing `identity:create_grant(project_id=08ee65f8b338491f80e7beac5e316b23, user_id=f80d59fec4994e689e857ba2087d3a23, role_id=1f7cf26664d04e5e9331ead293476442)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:51.966239 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16973409-6295-4da7-b673-5ebfa85c852d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:51.998584 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 309/604] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:53:51 2022] PUT /identity/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/f80d59fec4994e689e857ba2087d3a23/roles/1f7cf26664d04e5e9331ead293476442 => generated 0 bytes in 266 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:53:52.001738 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e75f44-c724-4d10-8011-2d0fcaa1052a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:52.002131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e75f44-c724-4d10-8011-2d0fcaa1052a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:52.002566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e75f44-c724-4d10-8011-2d0fcaa1052a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:52.004540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 310/605] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:52.005762 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb7708-ee24-4287-b3e0-3fffc95fe82d None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:52.006056 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb7708-ee24-4287-b3e0-3fffc95fe82d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:52.006376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cb7708-ee24-4287-b3e0-3fffc95fe82d None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:52.012454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 311/606] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:52.012454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac51446-41df-4bcd-836b-30a3baabf38c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:52.012454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac51446-41df-4bcd-836b-30a3baabf38c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:52.012454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac51446-41df-4bcd-836b-30a3baabf38c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:52.049555 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3ac51446-41df-4bcd-836b-30a3baabf38c None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:52.049555 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3ac51446-41df-4bcd-836b-30a3baabf38c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:52.066787 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:52.078591 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:52.105471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e21a98-1bd9-477c-b277-ba8c3eed896f None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:52.157487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5caac-b327-44b3-9488-3f61570d5695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:52.159514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5caac-b327-44b3-9488-3f61570d5695 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:52.160119 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5caac-b327-44b3-9488-3f61570d5695 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:52.160623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a5caac-b327-44b3-9488-3f61570d5695 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:52.171422 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a5caac-b327-44b3-9488-3f61570d5695 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:52.174000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a5caac-b327-44b3-9488-3f61570d5695 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:52.177103 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 312/607] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:52.178528 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02af7ea-a6a4-4e66-a92d-87ee7e3e103f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:52.178822 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02af7ea-a6a4-4e66-a92d-87ee7e3e103f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:52.179098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02af7ea-a6a4-4e66-a92d-87ee7e3e103f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:52.202939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-c1a5caac-b327-44b3-9488-3f61570d5695 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:52.202939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:52.202939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:52.202939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:52.202939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:52.202939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:52.202939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:52.202939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:52.202939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:52.202939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:52.202939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:52.202939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:52.202939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:53:52.202939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:53:52.202939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:53:52.203927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:53:52.203927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:52.203927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:52.203927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:52.203927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:52.203927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:52.203927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:52.203927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:52.203927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:52.203927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:52.203927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:52.203927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:52.203927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:52.203927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:52.203927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:52.203927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:52.203927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:52.204805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:53:52.204805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:53:52.204805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:53:52.204805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:53:52.204805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:53:52.204805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:53:52.204805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:52.204805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:52.204805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 296/608] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:52.211491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5caac-b327-44b3-9488-3f61570d5695 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:52.216216 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b02af7ea-a6a4-4e66-a92d-87ee7e3e103f None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:52.217426 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b02af7ea-a6a4-4e66-a92d-87ee7e3e103f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:52.322031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 313/609] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:52.323078 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02af7ea-a6a4-4e66-a92d-87ee7e3e103f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:52.328961 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a5caac-b327-44b3-9488-3f61570d5695 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:52.371610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77aa16-f675-41b2-9504-30553da7b2a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:52.372685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77aa16-f675-41b2-9504-30553da7b2a8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:52.373373 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77aa16-f675-41b2-9504-30553da7b2a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:52.373921 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de77aa16-f675-41b2-9504-30553da7b2a8 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:52.375272 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77aa16-f675-41b2-9504-30553da7b2a8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:52.375626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77aa16-f675-41b2-9504-30553da7b2a8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:52.377086 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de77aa16-f675-41b2-9504-30553da7b2a8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:52.390731 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:52.396913 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 297/610] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:53:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:52.403622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77aa16-f675-41b2-9504-30553da7b2a8 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:52.431453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:52.462968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02af7ea-a6a4-4e66-a92d-87ee7e3e103f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:52.500893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b85522-c239-4088-8c60-650108cff637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:52.501609 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b85522-c239-4088-8c60-650108cff637 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:52.501999 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b85522-c239-4088-8c60-650108cff637 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:52.502449 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16b85522-c239-4088-8c60-650108cff637 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:52.514504 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16b85522-c239-4088-8c60-650108cff637 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:52.523211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-16b85522-c239-4088-8c60-650108cff637 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:52.523211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:52.523211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:52.523211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:52.523211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:52.523211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:52.523211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:52.523211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:52.523211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:52.523211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:52.523211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:52.523211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:52.523211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:52.523211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:52.523211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:52.523211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:52.524440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:52.524440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:52.524440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:52.524440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:52.524440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:52.524440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:52.524440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:52.524440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:52.524440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:52.524440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:52.524440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:52.524440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:52.524440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:52.524440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:52.524440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:52.524440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:52.525420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:52.525420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:52.525420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:52.525420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:52.525420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:52.525420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:52.525420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:52.525420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:52.525420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:52.525420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:52.525420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:52.525420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:52.526752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 314/611] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:53:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:52.527659 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b85522-c239-4088-8c60-650108cff637 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:52.527836 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de77aa16-f675-41b2-9504-30553da7b2a8 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:52.568378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ea04d-57b1-400e-be5a-9bba1e4b2ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:52.569517 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ea04d-57b1-400e-be5a-9bba1e4b2ccd None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:52.569976 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ea04d-57b1-400e-be5a-9bba1e4b2ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:52.570490 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504ea04d-57b1-400e-be5a-9bba1e4b2ccd None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:52.583359 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504ea04d-57b1-400e-be5a-9bba1e4b2ccd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:52.592550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-504ea04d-57b1-400e-be5a-9bba1e4b2ccd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:52.592550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:52.592550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:52.592550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:52.592550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:52.592550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:52.592550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:52.592550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:52.592550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:52.592550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:52.592550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:52.592550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:52.592550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:52.592550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:52.592550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:52.592550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:52.593534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:52.593534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:52.593534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:52.593534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:52.593534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:52.593534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:52.593534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:52.593534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:52.593534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:52.593534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:52.593534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:52.593534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:52.593534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:52.593534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:52.593534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:52.593534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:52.594518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:52.594518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:52.594518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:52.594518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:52.594518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:52.594518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:52.594518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:52.594518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:52.594518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:52.594518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:52.594518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:52.594518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:52.601691 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 298/612] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:53:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:52.602967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ea04d-57b1-400e-be5a-9bba1e4b2ccd None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:52.641236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16b85522-c239-4088-8c60-650108cff637 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:52.670690 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:52.677195 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd88e7b-51f4-4bb2-8cb6-9cb4d227c1d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:52.678450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd88e7b-51f4-4bb2-8cb6-9cb4d227c1d2 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:52.678647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd88e7b-51f4-4bb2-8cb6-9cb4d227c1d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:52.679022 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecd88e7b-51f4-4bb2-8cb6-9cb4d227c1d2 None admin] PATH_INFO: `/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/ce1c85ee44ef45d2b631e51112b5ea0a/roles/1f7cf26664d04e5e9331ead293476442` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:52.682227 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:52.697666 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd88e7b-51f4-4bb2-8cb6-9cb4d227c1d2 None admin] RBAC: Authorizing `identity:create_grant(project_id=08ee65f8b338491f80e7beac5e316b23, user_id=ce1c85ee44ef45d2b631e51112b5ea0a, role_id=1f7cf26664d04e5e9331ead293476442)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:52.699374 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecd88e7b-51f4-4bb2-8cb6-9cb4d227c1d2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:52.724598 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 315/613] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:53:52 2022] PUT /identity/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/ce1c85ee44ef45d2b631e51112b5ea0a/roles/1f7cf26664d04e5e9331ead293476442 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:53:52.725513 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd88e7b-51f4-4bb2-8cb6-9cb4d227c1d2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:52.748083 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504ea04d-57b1-400e-be5a-9bba1e4b2ccd None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:52.828743 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dff64f-5c01-4d15-81ad-22315fb06953 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:52.829711 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dff64f-5c01-4d15-81ad-22315fb06953 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:52.830175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dff64f-5c01-4d15-81ad-22315fb06953 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:52.830635 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dff64f-5c01-4d15-81ad-22315fb06953 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:52.831579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dff64f-5c01-4d15-81ad-22315fb06953 None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:52.832049 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dff64f-5c01-4d15-81ad-22315fb06953 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:52.833651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86dff64f-5c01-4d15-81ad-22315fb06953 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:52.844000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 299/614] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:53:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:52.850762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dff64f-5c01-4d15-81ad-22315fb06953 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:52.951010 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecd88e7b-51f4-4bb2-8cb6-9cb4d227c1d2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:52.991254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248fe36-7c1e-45e5-ae35-698d6d194b8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:52.993160 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248fe36-7c1e-45e5-ae35-698d6d194b8f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:52.993638 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248fe36-7c1e-45e5-ae35-698d6d194b8f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:52.994059 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248fe36-7c1e-45e5-ae35-698d6d194b8f None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:52.995006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248fe36-7c1e-45e5-ae35-698d6d194b8f None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:52.995457 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248fe36-7c1e-45e5-ae35-698d6d194b8f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:52.996972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248fe36-7c1e-45e5-ae35-698d6d194b8f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:53.010847 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 316/615] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:53:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:53.016573 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248fe36-7c1e-45e5-ae35-698d6d194b8f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:53.059200 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86dff64f-5c01-4d15-81ad-22315fb06953 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:53.092855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:53.112406 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:53.127078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41231c5-475b-4ef3-aab1-c03545c38b0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:53.127886 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41231c5-475b-4ef3-aab1-c03545c38b0a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:53.128170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41231c5-475b-4ef3-aab1-c03545c38b0a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:53.128447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e41231c5-475b-4ef3-aab1-c03545c38b0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:53.147191 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41231c5-475b-4ef3-aab1-c03545c38b0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:53.148663 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e41231c5-475b-4ef3-aab1-c03545c38b0a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:53.188749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-e41231c5-475b-4ef3-aab1-c03545c38b0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:53.188749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:53.188749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:53.188749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:53.188749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:53.188749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:53.188749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:53.188749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:53.188749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:53.188749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:53.188749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:53.188749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:53.188749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:53.188749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:53.188749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:53.189704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:53.189704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:53.189704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:53.189704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:53.189704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:53.189704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:53.189704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:53.189704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:53.189704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:53.189704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:53.189704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:53.189704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:53.189704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:53.189704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:53.189704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:53.189704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:53.189704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:53.190640 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:53.190640 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:53.190640 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:53.190640 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:53.194329 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 300/616] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 341 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:53:53.195961 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41231c5-475b-4ef3-aab1-c03545c38b0a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:53.228256 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248fe36-7c1e-45e5-ae35-698d6d194b8f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:53.324978 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a0d32-348d-4905-ba60-ff48685a86fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:53.326101 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a0d32-348d-4905-ba60-ff48685a86fa None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:53.326639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a0d32-348d-4905-ba60-ff48685a86fa None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:53.327208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a0d32-348d-4905-ba60-ff48685a86fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:53.351960 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a0d32-348d-4905-ba60-ff48685a86fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:53.353433 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a0d32-348d-4905-ba60-ff48685a86fa None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:53.386777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-f76a0d32-348d-4905-ba60-ff48685a86fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:53.386777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:53.386777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:53.386777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:53.386777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:53.386777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:53.386777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:53.386777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:53.386777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:53.386777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:53.386777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:53.386777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:53.386777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:53.386777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:53.386777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:53.387731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:53.387731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:53.387731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:53.387731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:53.387731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:53.387731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:53.387731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:53.387731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:53.387731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:53.387731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:53.387731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:53.387731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:53.387731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:53.387731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:53.387731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:53.387731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:53.387731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:53.388577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:53.388577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:53.388577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:53.388577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:53.390922 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 317/617] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 375 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:53.401327 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a0d32-348d-4905-ba60-ff48685a86fa None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:53.438598 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e41231c5-475b-4ef3-aab1-c03545c38b0a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:53.506875 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e93dd-fde1-4c9d-afcb-64e43b87ef11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:53.507532 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e93dd-fde1-4c9d-afcb-64e43b87ef11 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:53.507820 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e93dd-fde1-4c9d-afcb-64e43b87ef11 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:53.508090 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536e93dd-fde1-4c9d-afcb-64e43b87ef11 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:53.508842 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e93dd-fde1-4c9d-afcb-64e43b87ef11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:53.509149 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e93dd-fde1-4c9d-afcb-64e43b87ef11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:53.510596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536e93dd-fde1-4c9d-afcb-64e43b87ef11 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:53.528763 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 301/618] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:53.534568 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e93dd-fde1-4c9d-afcb-64e43b87ef11 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:53.602225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:53.610953 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:53.620795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a0d32-348d-4905-ba60-ff48685a86fa None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:53.692671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a170f2-67a3-4773-be12-a47badf10a96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:53.693412 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a170f2-67a3-4773-be12-a47badf10a96 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:53.693699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a170f2-67a3-4773-be12-a47badf10a96 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:53.693974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a170f2-67a3-4773-be12-a47badf10a96 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:53.694805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a170f2-67a3-4773-be12-a47badf10a96 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:53.695117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a170f2-67a3-4773-be12-a47badf10a96 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:53.696519 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54a170f2-67a3-4773-be12-a47badf10a96 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:53.728315 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 318/619] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 328 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:53.731534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a170f2-67a3-4773-be12-a47badf10a96 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:53.751046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536e93dd-fde1-4c9d-afcb-64e43b87ef11 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:53.797903 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c6149-2539-436a-841b-73b5b87ff951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:53.798659 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c6149-2539-436a-841b-73b5b87ff951 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:53.798942 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c6149-2539-436a-841b-73b5b87ff951 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:53.799221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225c6149-2539-436a-841b-73b5b87ff951 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:53.816984 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c6149-2539-436a-841b-73b5b87ff951 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:53.818596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225c6149-2539-436a-841b-73b5b87ff951 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:53.850420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-225c6149-2539-436a-841b-73b5b87ff951 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 13:53:53.850420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:53.850420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:53.850420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:53.850420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:53.850420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:53.850420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:53.850420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:53.850420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:53.850420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:53.850420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:53.850420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:53.850420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:53:53.850420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:53:53.850420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:53:53.850420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:53:53.851740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:53.851740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:53.851740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:53:53.851740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:53.851740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:53:53.851740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:53.851740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:53.851740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:53.851740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:53.851740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:53.851740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:53.851740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:53.851740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:53.851740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:53.851740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:53.851740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:53.851740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:53.851740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:53.852675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:53.852675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:53.852675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:53:53.852675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:53:53.852675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:53:53.852675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:53:53.852675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:53:53.852675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:53:53.852675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 13:53:53.852675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:53.852675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 302/620] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Jul 11 13:53:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:53.860379 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c6149-2539-436a-841b-73b5b87ff951 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:53.890271 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54a170f2-67a3-4773-be12-a47badf10a96 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:53.930481 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68359f1b-2cd8-4e7f-92c4-7729e5339842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:53.931669 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68359f1b-2cd8-4e7f-92c4-7729e5339842 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:53.932174 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68359f1b-2cd8-4e7f-92c4-7729e5339842 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:53.933160 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68359f1b-2cd8-4e7f-92c4-7729e5339842 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:53.942455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68359f1b-2cd8-4e7f-92c4-7729e5339842 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:53.944229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68359f1b-2cd8-4e7f-92c4-7729e5339842 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:53.962385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-68359f1b-2cd8-4e7f-92c4-7729e5339842 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 13:53:53.962385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:53.962385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:53.962385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:53.962385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:53.962385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:53.962385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:53.962385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:53.962385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:53.962385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:53.962385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:53.962385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:53.962385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:53:53.962385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:53:53.962385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:53:53.962385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:53:53.963391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:53.963391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:53.963391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:53:53.963391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:53.963391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:53:53.963391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:53.963391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:53.963391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:53.963391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:53.963391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:53.963391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:53.963391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:53.963391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:53.963391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:53.963391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:53.963391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:53.963391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:53.963391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:53.964289 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:53.964289 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:53.964289 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:53:53.964289 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:53:53.964289 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:53:53.964289 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:53:53.964289 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:53:53.964289 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:53:53.964289 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 13:53:53.964289 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:53.967644 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 319/621] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:53:53 2022] GET /identity/v3/users/swift => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:53.976032 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68359f1b-2cd8-4e7f-92c4-7729e5339842 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:53.998603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:54.096949 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68359f1b-2cd8-4e7f-92c4-7729e5339842 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:54.106595 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:54.142325 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225c6149-2539-436a-841b-73b5b87ff951 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:54.155193 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2656fd-1bfc-4d45-b5b1-a1f1ff25722d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:54.155908 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2656fd-1bfc-4d45-b5b1-a1f1ff25722d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:54.156196 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2656fd-1bfc-4d45-b5b1-a1f1ff25722d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:54.156486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2656fd-1bfc-4d45-b5b1-a1f1ff25722d None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:54.157369 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2656fd-1bfc-4d45-b5b1-a1f1ff25722d None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:54.157682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2656fd-1bfc-4d45-b5b1-a1f1ff25722d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:54.159204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2656fd-1bfc-4d45-b5b1-a1f1ff25722d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:54.190607 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 320/622] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Jul 11 13:53:53 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:54.194391 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdfd7f-2373-49f3-85f9-a899642c3fcd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:54.195297 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdfd7f-2373-49f3-85f9-a899642c3fcd None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:54.195788 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdfd7f-2373-49f3-85f9-a899642c3fcd None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:54.196854 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2656fd-1bfc-4d45-b5b1-a1f1ff25722d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:54.197329 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cbdfd7f-2373-49f3-85f9-a899642c3fcd None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:54.198898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdfd7f-2373-49f3-85f9-a899642c3fcd None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:54.199772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdfd7f-2373-49f3-85f9-a899642c3fcd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:54.203768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cbdfd7f-2373-49f3-85f9-a899642c3fcd None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:54.233827 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 303/623] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Jul 11 13:53:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:53:54.240688 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdfd7f-2373-49f3-85f9-a899642c3fcd None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:54.342142 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:54.368057 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2656fd-1bfc-4d45-b5b1-a1f1ff25722d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:54.384008 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cbdfd7f-2373-49f3-85f9-a899642c3fcd None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:54.405788 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aaeb1f1a-7169-4cfd-848e-5aec8ca16fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:54.407084 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb1f1a-7169-4cfd-848e-5aec8ca16fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:54.407609 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb1f1a-7169-4cfd-848e-5aec8ca16fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:54.408072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeb1f1a-7169-4cfd-848e-5aec8ca16fb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:54.417226 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee10c0fc-ba14-403b-9910-3a908998b965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:54.417874 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10c0fc-ba14-403b-9910-3a908998b965 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:54.418157 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10c0fc-ba14-403b-9910-3a908998b965 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:54.418462 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee10c0fc-ba14-403b-9910-3a908998b965 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:54.420009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb1f1a-7169-4cfd-848e-5aec8ca16fb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:54.422497 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aaeb1f1a-7169-4cfd-848e-5aec8ca16fb7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:54.439234 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:54.443747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-aaeb1f1a-7169-4cfd-848e-5aec8ca16fb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:54.443747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:54.443747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:54.443747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:54.443747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:54.443747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:54.443747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:54.443747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:54.443747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:54.443747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:54.443747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:54.443747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:54.443747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:54.443747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:54.443747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:54.444656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:54.444656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:54.444656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:54.444656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:54.444656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:54.444656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:54.444656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:54.444656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:54.444656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:54.444656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:54.444656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:54.444656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:54.444656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:54.444656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:54.444656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:54.444656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:54.444656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:54.445552 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:54.445552 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:54.445552 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:54.445552 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:54.445552 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 321/624] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:54.446252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1eaee5-0407-48c1-b63e-f3181f9e745c None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:54.446555 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1eaee5-0407-48c1-b63e-f3181f9e745c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:54.446832 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1eaee5-0407-48c1-b63e-f3181f9e745c None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:54.448250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 322/625] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:54.449162 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f84212-7f95-4c71-b33d-a22694afc433 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:54.449762 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f84212-7f95-4c71-b33d-a22694afc433 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:54.450025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f84212-7f95-4c71-b33d-a22694afc433 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:54.454433 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 323/626] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:54.454433 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f84212-7f95-4c71-b33d-a22694afc433 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:54.502627 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee10c0fc-ba14-403b-9910-3a908998b965 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:54.504867 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee10c0fc-ba14-403b-9910-3a908998b965 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:54.560199 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f84212-7f95-4c71-b33d-a22694afc433 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:54.588339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:54.600292 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0627902-82d5-4a56-99f8-eb167df57bd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:54.601200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0627902-82d5-4a56-99f8-eb167df57bd4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:54.601714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0627902-82d5-4a56-99f8-eb167df57bd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:54.601932 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0627902-82d5-4a56-99f8-eb167df57bd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:54.602678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0627902-82d5-4a56-99f8-eb167df57bd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:54.603165 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0627902-82d5-4a56-99f8-eb167df57bd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:54.604356 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0627902-82d5-4a56-99f8-eb167df57bd4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:54.622354 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 324/627] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:54.622354 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5c99b-a4cc-4302-93f5-4a5f1af17a24 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:54.622354 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5c99b-a4cc-4302-93f5-4a5f1af17a24 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:54.622354 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26a5c99b-a4cc-4302-93f5-4a5f1af17a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:54.631600 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 304/628] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:54 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:54.636082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a7749-ee51-41d9-a1d9-72cd4d978dd6 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:54.636082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a7749-ee51-41d9-a1d9-72cd4d978dd6 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:54.636082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199a7749-ee51-41d9-a1d9-72cd4d978dd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:54.642521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-26a5c99b-a4cc-4302-93f5-4a5f1af17a24 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:54.643730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-26a5c99b-a4cc-4302-93f5-4a5f1af17a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:54.651168 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-199a7749-ee51-41d9-a1d9-72cd4d978dd6 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:54.652423 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-199a7749-ee51-41d9-a1d9-72cd4d978dd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:54.705265 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:54.730345 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 325/629] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:54.730345 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5c99b-a4cc-4302-93f5-4a5f1af17a24 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:54.734102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 305/630] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:54.735282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a7749-ee51-41d9-a1d9-72cd4d978dd6 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:54.861596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26a5c99b-a4cc-4302-93f5-4a5f1af17a24 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:54.863989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199a7749-ee51-41d9-a1d9-72cd4d978dd6 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:54.883262 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:54.903976 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b31fd-cf4e-4537-80c6-a725fb34c42f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:54.904202 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153bff3e-60c0-4da3-8789-d9fb945c8229 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:54.904795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b31fd-cf4e-4537-80c6-a725fb34c42f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:54.905586 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b31fd-cf4e-4537-80c6-a725fb34c42f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:54.905862 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-551b31fd-cf4e-4537-80c6-a725fb34c42f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:54.906174 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153bff3e-60c0-4da3-8789-d9fb945c8229 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:54.906817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153bff3e-60c0-4da3-8789-d9fb945c8229 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:54.907268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-153bff3e-60c0-4da3-8789-d9fb945c8229 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:54.914048 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b31fd-cf4e-4537-80c6-a725fb34c42f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:54.915550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-551b31fd-cf4e-4537-80c6-a725fb34c42f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:54.919988 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153bff3e-60c0-4da3-8789-d9fb945c8229 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:54.922353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-153bff3e-60c0-4da3-8789-d9fb945c8229 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:54.931419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-551b31fd-cf4e-4537-80c6-a725fb34c42f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:54.931419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:54.931419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:54.931419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:54.931419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:54.931419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:54.931419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:54.931419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:54.931419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:54.931419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:54.931419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:54.931419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:54.931419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:53:54.931419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:53:54.931419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:53:54.932421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:53:54.932421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:54.932421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:54.932421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:54.932421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:54.932421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:54.932421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:54.932421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:54.932421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:54.932421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:54.932421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:54.932421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:54.932421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:54.932421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:54.932421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:54.932421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:54.932421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:54.938544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:53:54.938544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:53:54.938544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:53:54.938544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:53:54.938544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:53:54.938544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:53:54.938544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:54.938544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:54.938544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 326/631] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:54.938544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b31fd-cf4e-4537-80c6-a725fb34c42f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:54.946134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-153bff3e-60c0-4da3-8789-d9fb945c8229 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:54.946134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:54.946134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:54.946134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:54.946134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:54.946134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:54.946134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:54.946134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:54.946134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:54.946134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:54.946134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:54.946134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:54.946134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:54.946134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:54.946134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:54.947169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:54.947169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:54.947169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:54.947169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:54.947169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:54.947169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:54.947169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:54.947169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:54.947169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:54.947169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:54.947169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:54.947169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:54.947169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:54.947169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:54.947169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:54.947169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:54.947169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:54.948048 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:54.948048 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:54.948048 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:54.948048 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:54.948048 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 306/632] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:54.953481 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153bff3e-60c0-4da3-8789-d9fb945c8229 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:54.988919 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:55.072080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-551b31fd-cf4e-4537-80c6-a725fb34c42f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:55.110639 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-153bff3e-60c0-4da3-8789-d9fb945c8229 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:55.112937 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc420cda-512f-4760-8b32-b85086ae9216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:55.113678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc420cda-512f-4760-8b32-b85086ae9216 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:55.113953 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc420cda-512f-4760-8b32-b85086ae9216 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:55.114235 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc420cda-512f-4760-8b32-b85086ae9216 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:55.130123 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc420cda-512f-4760-8b32-b85086ae9216 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:55.146001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-fc420cda-512f-4760-8b32-b85086ae9216 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:55.146001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:55.146001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:55.146001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:55.146001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:55.146001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:55.146001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:55.146001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:55.146001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:55.146001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:55.146001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:55.146001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:55.146001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:55.146001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:55.146001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:55.146001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:55.147243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:55.147243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:55.147243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:55.147243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:55.147243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:55.147243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:55.147243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:55.147243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:55.147243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:55.147243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:55.147243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:55.147243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:55.147243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:55.147243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:55.147243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:55.147243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:55.148204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:55.148204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:55.148204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:55.148204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:55.148204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:55.148204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:55.148204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:55.148204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:55.148204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:55.148204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:55.148204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:55.148204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:55.152930 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15126d66-e211-4038-8f7b-b0dfc1168a58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:55.153960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15126d66-e211-4038-8f7b-b0dfc1168a58 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:55.154460 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15126d66-e211-4038-8f7b-b0dfc1168a58 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:55.154994 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15126d66-e211-4038-8f7b-b0dfc1168a58 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:55.155812 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 327/633] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:53:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:55.157241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc420cda-512f-4760-8b32-b85086ae9216 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:55.168040 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15126d66-e211-4038-8f7b-b0dfc1168a58 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:55.176309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-15126d66-e211-4038-8f7b-b0dfc1168a58 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:55.176309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:55.176309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:55.176309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:55.176309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:55.176309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:55.176309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:55.176309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:55.176309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:55.176309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:55.176309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:55.176309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:55.176309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:55.176309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:55.176309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:55.176309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:55.177497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:55.177497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:55.177497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:55.177497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:55.177497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:55.177497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:55.177497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:55.177497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:55.177497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:55.177497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:55.177497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:55.177497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:55.177497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:55.177497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:55.177497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:55.177497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:55.178777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:55.178777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:55.178777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:55.178777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:55.178777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:55.178777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:55.178777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:55.178777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:55.178777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:55.178777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:55.178777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:55.178777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:55.178777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 307/634] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:53:54 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:55.178777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15126d66-e211-4038-8f7b-b0dfc1168a58 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:55.210958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:55.283602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc420cda-512f-4760-8b32-b85086ae9216 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:55.289927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:55.300377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15126d66-e211-4038-8f7b-b0dfc1168a58 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:55.319796 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e3bd3-339e-42a5-a0eb-64311be5091d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:55.320469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e3bd3-339e-42a5-a0eb-64311be5091d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:55.320755 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e3bd3-339e-42a5-a0eb-64311be5091d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:55.321031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-575e3bd3-339e-42a5-a0eb-64311be5091d None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:55.321829 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e3bd3-339e-42a5-a0eb-64311be5091d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:55.322137 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e3bd3-339e-42a5-a0eb-64311be5091d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:55.323597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-575e3bd3-339e-42a5-a0eb-64311be5091d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:55.336051 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 328/635] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:53:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:55.336360 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a0158-bd8c-4f6e-95dc-94c822354df8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:55.336874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e3bd3-339e-42a5-a0eb-64311be5091d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:55.337047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a0158-bd8c-4f6e-95dc-94c822354df8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:55.337303 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a0158-bd8c-4f6e-95dc-94c822354df8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:55.337571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2a0158-bd8c-4f6e-95dc-94c822354df8 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:55.338326 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a0158-bd8c-4f6e-95dc-94c822354df8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:55.338631 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a0158-bd8c-4f6e-95dc-94c822354df8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:55.340011 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2a0158-bd8c-4f6e-95dc-94c822354df8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:55.353622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 308/636] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:55.354957 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a0158-bd8c-4f6e-95dc-94c822354df8 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:55.445186 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:55.449766 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-575e3bd3-339e-42a5-a0eb-64311be5091d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:55.466770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2a0158-bd8c-4f6e-95dc-94c822354df8 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:55.485128 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a2cce6e-5156-4375-8bbd-36e445b52f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:55.486076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cce6e-5156-4375-8bbd-36e445b52f7c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:55.486387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cce6e-5156-4375-8bbd-36e445b52f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:55.486666 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a2cce6e-5156-4375-8bbd-36e445b52f7c None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:55.487417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cce6e-5156-4375-8bbd-36e445b52f7c None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:55.487723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cce6e-5156-4375-8bbd-36e445b52f7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:55.489089 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a2cce6e-5156-4375-8bbd-36e445b52f7c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:55.497678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 329/637] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:53:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:55.499516 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac9018-69ed-4321-8104-a38c2efd3654 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:55.499797 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac9018-69ed-4321-8104-a38c2efd3654 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:55.500073 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ac9018-69ed-4321-8104-a38c2efd3654 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:55.501529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 330/638] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:55.503820 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d16bd-db42-4b3c-80b8-fee6ee1a1dc9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:55.504511 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d16bd-db42-4b3c-80b8-fee6ee1a1dc9 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:55.504795 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d16bd-db42-4b3c-80b8-fee6ee1a1dc9 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:55.505065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-140d16bd-db42-4b3c-80b8-fee6ee1a1dc9 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:55.505826 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d16bd-db42-4b3c-80b8-fee6ee1a1dc9 None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:55.506135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d16bd-db42-4b3c-80b8-fee6ee1a1dc9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:55.506335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac9018-69ed-4321-8104-a38c2efd3654 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:55.507566 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-140d16bd-db42-4b3c-80b8-fee6ee1a1dc9 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:55.516981 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 309/639] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:53:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:55.517819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d16bd-db42-4b3c-80b8-fee6ee1a1dc9 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:55.534909 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:55.614831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ac9018-69ed-4321-8104-a38c2efd3654 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:55.624640 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-140d16bd-db42-4b3c-80b8-fee6ee1a1dc9 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:55.650374 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd2882-fb48-4a19-a89f-3c2797f7feec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:55.651285 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd2882-fb48-4a19-a89f-3c2797f7feec None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:55.651572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd2882-fb48-4a19-a89f-3c2797f7feec None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:55.651851 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdd2882-fb48-4a19-a89f-3c2797f7feec None admin] PATH_INFO: `/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/d9c1369033a44c1c81d4c1c9ff2bbedb/roles/1f7cf26664d04e5e9331ead293476442` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:55.659573 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a958c3a-d05c-421d-bb88-eb0f30015072 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:55.660377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a958c3a-d05c-421d-bb88-eb0f30015072 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:55.660653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a958c3a-d05c-421d-bb88-eb0f30015072 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:55.660927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a958c3a-d05c-421d-bb88-eb0f30015072 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:55.667294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd2882-fb48-4a19-a89f-3c2797f7feec None admin] RBAC: Authorizing `identity:create_grant(project_id=08ee65f8b338491f80e7beac5e316b23, user_id=d9c1369033a44c1c81d4c1c9ff2bbedb, role_id=1f7cf26664d04e5e9331ead293476442)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:55.667936 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a958c3a-d05c-421d-bb88-eb0f30015072 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:55.668901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdd2882-fb48-4a19-a89f-3c2797f7feec None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:55.669370 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a958c3a-d05c-421d-bb88-eb0f30015072 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:55.682525 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 331/640] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:53:55 2022] PUT /identity/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/d9c1369033a44c1c81d4c1c9ff2bbedb/roles/1f7cf26664d04e5e9331ead293476442 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:53:55.683349 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd2882-fb48-4a19-a89f-3c2797f7feec None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:55.684580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-4a958c3a-d05c-421d-bb88-eb0f30015072 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:55.684580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:55.684580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:55.684580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:55.684580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:55.684580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:55.684580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:55.684580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:55.684580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:55.684580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:55.684580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:55.684580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:55.684580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:53:55.684580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:53:55.684580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:53:55.685718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:53:55.685718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:55.685718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:55.685718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:55.685718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:55.685718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:55.685718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:55.685718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:55.685718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:55.685718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:55.685718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:55.685718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:55.685718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:55.685718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:55.685718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:55.685718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:55.685718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:55.687293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:53:55.687293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:53:55.687293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:53:55.687293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:53:55.687293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:53:55.687293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:53:55.687293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:55.687293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:55.687293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 310/641] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:55.688679 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad9461-1b65-4991-aa01-00554fe3b0dd None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:55.688974 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad9461-1b65-4991-aa01-00554fe3b0dd None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:55.689248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdad9461-1b65-4991-aa01-00554fe3b0dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:55.706663 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:55.717504 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-bdad9461-1b65-4991-aa01-00554fe3b0dd None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:55.719087 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bdad9461-1b65-4991-aa01-00554fe3b0dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:55.807667 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 311/642] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:55.809525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad9461-1b65-4991-aa01-00554fe3b0dd None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:55.816219 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:55.830957 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdd2882-fb48-4a19-a89f-3c2797f7feec None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:55.871781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3913e-6dad-4109-b8a9-486e44bb169a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:55.872708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3913e-6dad-4109-b8a9-486e44bb169a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:55.873150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3913e-6dad-4109-b8a9-486e44bb169a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:55.873562 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38d3913e-6dad-4109-b8a9-486e44bb169a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:55.886996 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d3913e-6dad-4109-b8a9-486e44bb169a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:55.888470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38d3913e-6dad-4109-b8a9-486e44bb169a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:55.907424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-38d3913e-6dad-4109-b8a9-486e44bb169a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:55.907424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:55.907424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:55.907424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:55.907424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:55.907424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:55.907424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:55.907424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:55.907424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:55.907424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:55.907424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:55.907424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:55.907424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:55.907424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:55.907424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:55.909170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:55.909170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:55.909170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:55.909170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:55.909170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:55.909170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:55.909170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:55.909170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:55.909170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:55.909170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:55.909170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:55.909170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:55.909170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:55.909170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:55.909170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:55.909170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:55.909170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:55.910918 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:55.910918 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:55.910918 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:55.910918 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:55.910918 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 332/643] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:55.910918 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3913e-6dad-4109-b8a9-486e44bb169a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:55.968264 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdad9461-1b65-4991-aa01-00554fe3b0dd None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:55.980839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:56.006628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988e984-f2f7-4fcd-9b69-1661e4f1d425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:56.007453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988e984-f2f7-4fcd-9b69-1661e4f1d425 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:56.007744 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988e984-f2f7-4fcd-9b69-1661e4f1d425 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:56.008027 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8988e984-f2f7-4fcd-9b69-1661e4f1d425 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:56.020089 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988e984-f2f7-4fcd-9b69-1661e4f1d425 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:56.021584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8988e984-f2f7-4fcd-9b69-1661e4f1d425 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:56.025775 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38d3913e-6dad-4109-b8a9-486e44bb169a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:56.041319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-8988e984-f2f7-4fcd-9b69-1661e4f1d425 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:56.041319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:56.041319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:56.041319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:56.041319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:56.041319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:56.041319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:56.041319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:56.041319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:56.041319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:56.041319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:56.041319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:56.041319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:56.041319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:56.041319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:56.042334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:56.042334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:56.042334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:56.042334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:56.042334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:56.042334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:56.042334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:56.042334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:56.042334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:56.042334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:56.042334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:56.042334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:56.042334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:56.042334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:56.042334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:56.042334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:56.042334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:56.043395 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:56.043395 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:56.043395 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:56.043395 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:56.043395 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 312/644] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 234 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:56.043683 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988e984-f2f7-4fcd-9b69-1661e4f1d425 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:56.087387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de91be-a064-435a-a34d-d5add466ca99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:56.088513 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de91be-a064-435a-a34d-d5add466ca99 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:56.089000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de91be-a064-435a-a34d-d5add466ca99 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:56.089537 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78de91be-a064-435a-a34d-d5add466ca99 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:56.091079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de91be-a064-435a-a34d-d5add466ca99 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:56.091642 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de91be-a064-435a-a34d-d5add466ca99 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:56.094432 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78de91be-a064-435a-a34d-d5add466ca99 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:56.117307 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:56.117899 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 333/645] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:53:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:56.119391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de91be-a064-435a-a34d-d5add466ca99 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:56.173562 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8988e984-f2f7-4fcd-9b69-1661e4f1d425 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:56.213538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aafe3f-5dea-4977-bd9d-42d30a4c9792 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:56.214404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aafe3f-5dea-4977-bd9d-42d30a4c9792 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:56.214691 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aafe3f-5dea-4977-bd9d-42d30a4c9792 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:56.214969 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88aafe3f-5dea-4977-bd9d-42d30a4c9792 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:56.226386 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88aafe3f-5dea-4977-bd9d-42d30a4c9792 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:56.234603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-88aafe3f-5dea-4977-bd9d-42d30a4c9792 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:56.234603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:56.234603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:56.234603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:56.234603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:56.234603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:56.234603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:56.234603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:56.234603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:56.234603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:56.234603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:56.234603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:56.234603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:56.234603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:56.234603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:56.234603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:56.235605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:56.235605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:56.235605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:56.235605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:56.235605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:56.235605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:56.235605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:56.235605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:56.235605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:56.235605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:56.235605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:56.235605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:56.235605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:56.235605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:56.235605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:56.235605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:56.236459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:56.236459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:56.236459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:56.236459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:56.236459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:56.236459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:56.236459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:56.236459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:56.236459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:56.236459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:56.236459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:56.236459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:56.236459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 313/646] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:53:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:56.237170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aafe3f-5dea-4977-bd9d-42d30a4c9792 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:56.268415 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78de91be-a064-435a-a34d-d5add466ca99 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:56.295218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:56.309991 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff6dab-3bf6-422f-933d-3eafbe9b0be0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:56.310696 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff6dab-3bf6-422f-933d-3eafbe9b0be0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:56.310988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff6dab-3bf6-422f-933d-3eafbe9b0be0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:56.311308 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2ff6dab-3bf6-422f-933d-3eafbe9b0be0 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:56.312047 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff6dab-3bf6-422f-933d-3eafbe9b0be0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:56.312353 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff6dab-3bf6-422f-933d-3eafbe9b0be0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:56.313753 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2ff6dab-3bf6-422f-933d-3eafbe9b0be0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:56.326208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 334/647] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:56.327167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff6dab-3bf6-422f-933d-3eafbe9b0be0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:56.360014 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88aafe3f-5dea-4977-bd9d-42d30a4c9792 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:56.379462 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:56.399637 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b36d6f-1572-4a7f-8773-b90c1e2e5eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:56.400305 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b36d6f-1572-4a7f-8773-b90c1e2e5eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:56.400589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b36d6f-1572-4a7f-8773-b90c1e2e5eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:56.400862 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15b36d6f-1572-4a7f-8773-b90c1e2e5eb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:56.401593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b36d6f-1572-4a7f-8773-b90c1e2e5eb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:56.401926 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b36d6f-1572-4a7f-8773-b90c1e2e5eb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:56.403327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15b36d6f-1572-4a7f-8773-b90c1e2e5eb7 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:56.416727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 314/648] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:56.417555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b36d6f-1572-4a7f-8773-b90c1e2e5eb7 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:56.447119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2ff6dab-3bf6-422f-933d-3eafbe9b0be0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:56.482946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a571d50-7de8-4e34-a79f-491516fce3ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:56.484162 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a571d50-7de8-4e34-a79f-491516fce3ae None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:56.484449 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a571d50-7de8-4e34-a79f-491516fce3ae None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:56.484731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a571d50-7de8-4e34-a79f-491516fce3ae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:56.485586 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a571d50-7de8-4e34-a79f-491516fce3ae None admin] RBAC: Adding query filter params (role.id=1f7cf26664d04e5e9331ead293476442, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=ef5dac3f311e43809a46995bab56e780) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:56.485910 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a571d50-7de8-4e34-a79f-491516fce3ae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:56.487394 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a571d50-7de8-4e34-a79f-491516fce3ae None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:56.494100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 335/649] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:53:56 2022] GET /identity/v3/role_assignments?user.id=ef5dac3f311e43809a46995bab56e780&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=1f7cf26664d04e5e9331ead293476442 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:56.495035 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a571d50-7de8-4e34-a79f-491516fce3ae None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:56.548080 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15b36d6f-1572-4a7f-8773-b90c1e2e5eb7 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:56.554289 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:56.608931 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fdfb57-0ec6-4aa4-99df-2f70c69a2ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:56.610528 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdfb57-0ec6-4aa4-99df-2f70c69a2ae2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:56.611335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdfb57-0ec6-4aa4-99df-2f70c69a2ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:56.611847 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28fdfb57-0ec6-4aa4-99df-2f70c69a2ae2 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:56.613112 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fdfb57-0ec6-4aa4-99df-2f70c69a2ae2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:56.614238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fdfb57-0ec6-4aa4-99df-2f70c69a2ae2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:56.616660 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28fdfb57-0ec6-4aa4-99df-2f70c69a2ae2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:56.630405 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 315/650] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:53:56 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:56.632330 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fdfb57-0ec6-4aa4-99df-2f70c69a2ae2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:56.634951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a571d50-7de8-4e34-a79f-491516fce3ae None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:56.681297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c5994-806a-432f-ae4d-85696f2f4f87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:56.682871 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c5994-806a-432f-ae4d-85696f2f4f87 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:56.684334 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c5994-806a-432f-ae4d-85696f2f4f87 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:56.684811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0c5994-806a-432f-ae4d-85696f2f4f87 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:56.689798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:56.697463 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c5994-806a-432f-ae4d-85696f2f4f87 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:56.698979 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0c5994-806a-432f-ae4d-85696f2f4f87 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:56.716076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-4d0c5994-806a-432f-ae4d-85696f2f4f87 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 13:53:56.716076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:56.716076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:56.716076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:56.716076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:56.716076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:56.716076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:56.716076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:56.716076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:56.716076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:56.716076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:56.716076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:56.716076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:53:56.716076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:53:56.716076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:53:56.716076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:53:56.717041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:56.717041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:56.717041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:53:56.717041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:56.717041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:53:56.717041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:56.717041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:56.717041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:56.717041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:56.717041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:56.717041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:56.717041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:56.717041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:56.717041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:56.717041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:56.717041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:56.717041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:56.717041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:56.718733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:56.718733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:56.718733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:53:56.718733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:53:56.718733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:53:56.718733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:53:56.718733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:53:56.718733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:53:56.718733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 13:53:56.718733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:56.718733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 336/651] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:56 2022] GET /identity/v3/users/cinder => generated 84 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:56.723287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c5994-806a-432f-ae4d-85696f2f4f87 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:56.786483 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28fdfb57-0ec6-4aa4-99df-2f70c69a2ae2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:56.833038 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872d616-b6df-4726-b011-845d9072139b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:56.833840 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872d616-b6df-4726-b011-845d9072139b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:56.834470 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872d616-b6df-4726-b011-845d9072139b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:56.834470 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2872d616-b6df-4726-b011-845d9072139b None admin] PATH_INFO: `/v3/users/glance` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:56.842186 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2872d616-b6df-4726-b011-845d9072139b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:56.844337 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2872d616-b6df-4726-b011-845d9072139b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:56.858474 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0c5994-806a-432f-ae4d-85696f2f4f87 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:56.861946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-2872d616-b6df-4726-b011-845d9072139b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 11 13:53:56.861946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:56.861946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:56.861946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:56.861946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:56.861946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:56.861946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:56.861946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:56.861946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:56.861946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:56.861946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:56.861946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:56.861946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:53:56.861946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:53:56.861946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:53:56.861946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:53:56.862932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:56.862932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:56.862932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:53:56.862932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:56.862932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:53:56.862932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:56.862932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:56.862932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:56.862932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:56.862932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:56.862932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:56.862932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:56.862932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:56.862932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:56.862932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:56.862932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:56.862932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:56.862932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:56.864267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:56.864267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:56.864267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:53:56.864267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:53:56.864267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:53:56.864267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:53:56.864267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:53:56.864267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:53:56.864267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 11 13:53:56.864267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:56.864267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 316/652] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:56 2022] GET /identity/v3/users/glance => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:56.864267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872d616-b6df-4726-b011-845d9072139b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:56.870543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:56.916341 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb895ef4-c79e-44f3-94c0-b0eee990be4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:56.918633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb895ef4-c79e-44f3-94c0-b0eee990be4a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:56.918849 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb895ef4-c79e-44f3-94c0-b0eee990be4a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:56.919115 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb895ef4-c79e-44f3-94c0-b0eee990be4a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:56.937119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb895ef4-c79e-44f3-94c0-b0eee990be4a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:56.939337 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb895ef4-c79e-44f3-94c0-b0eee990be4a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:56.965578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-cb895ef4-c79e-44f3-94c0-b0eee990be4a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:56.965578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:56.965578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:56.965578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:56.965578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:56.965578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:56.965578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:56.965578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:56.965578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:56.965578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:56.965578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:56.965578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:56.965578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:56.965578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:56.965578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:56.966560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:56.966560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:56.966560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:56.966560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:56.966560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:56.966560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:56.966560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:56.966560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:56.966560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:56.966560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:56.966560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:56.966560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:56.966560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:56.966560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:56.966560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:56.966560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:56.966560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:56.967463 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:56.967463 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:56.967463 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:56.967463 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:56.970081 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 337/653] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:56.971493 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb895ef4-c79e-44f3-94c0-b0eee990be4a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:57.020170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:57.024676 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2872d616-b6df-4726-b011-845d9072139b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:57.069745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008eef0-11e0-4364-b14d-20157ec6e367 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:57.070422 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008eef0-11e0-4364-b14d-20157ec6e367 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:57.070713 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008eef0-11e0-4364-b14d-20157ec6e367 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:57.073938 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b008eef0-11e0-4364-b14d-20157ec6e367 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:57.074869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008eef0-11e0-4364-b14d-20157ec6e367 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:57.075433 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008eef0-11e0-4364-b14d-20157ec6e367 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:57.076832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b008eef0-11e0-4364-b14d-20157ec6e367 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:57.092520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb895ef4-c79e-44f3-94c0-b0eee990be4a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:57.102515 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 317/654] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Jul 11 13:53:56 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:57.103513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008eef0-11e0-4364-b14d-20157ec6e367 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:57.134131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c57b76-6b21-4626-8424-e53b961fcfbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:57.135223 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c57b76-6b21-4626-8424-e53b961fcfbd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:57.135713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c57b76-6b21-4626-8424-e53b961fcfbd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:57.136192 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c57b76-6b21-4626-8424-e53b961fcfbd None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:57.137306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c57b76-6b21-4626-8424-e53b961fcfbd None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:57.137824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c57b76-6b21-4626-8424-e53b961fcfbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:57.140056 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c57b76-6b21-4626-8424-e53b961fcfbd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:57.159381 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 338/655] 10.222.0.49 () {56 vars in 1137 bytes} [Mon Jul 11 13:53:56 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:57.160629 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c57b76-6b21-4626-8424-e53b961fcfbd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:57.180264 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:57.269275 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c57b76-6b21-4626-8424-e53b961fcfbd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:57.286006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b008eef0-11e0-4364-b14d-20157ec6e367 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:57.307074 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040066f-9230-4692-b812-7b030ed52989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:57.308126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040066f-9230-4692-b812-7b030ed52989 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:57.308610 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040066f-9230-4692-b812-7b030ed52989 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:57.309103 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3040066f-9230-4692-b812-7b030ed52989 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:57.361721 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bb6cf-85a1-49ea-8c7c-9bf24e2a1666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:57.362258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bb6cf-85a1-49ea-8c7c-9bf24e2a1666 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:57.362555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bb6cf-85a1-49ea-8c7c-9bf24e2a1666 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:57.362832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c8bb6cf-85a1-49ea-8c7c-9bf24e2a1666 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:57.363569 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bb6cf-85a1-49ea-8c7c-9bf24e2a1666 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:57.363878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bb6cf-85a1-49ea-8c7c-9bf24e2a1666 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:57.369398 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c8bb6cf-85a1-49ea-8c7c-9bf24e2a1666 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:57.385748 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040066f-9230-4692-b812-7b030ed52989 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:57.387641 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3040066f-9230-4692-b812-7b030ed52989 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:57.396791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 318/656] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:57.398036 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bb6cf-85a1-49ea-8c7c-9bf24e2a1666 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:57.408697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:57.438724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:57.545363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 339/657] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 385 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:57.546956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040066f-9230-4692-b812-7b030ed52989 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:57.592219 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c8bb6cf-85a1-49ea-8c7c-9bf24e2a1666 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:57.634083 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e3258-7ba6-4238-8780-c1d43826ab2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:57.634889 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e3258-7ba6-4238-8780-c1d43826ab2a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:57.635000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e3258-7ba6-4238-8780-c1d43826ab2a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:57.635238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a46e3258-7ba6-4238-8780-c1d43826ab2a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:57.723171 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3040066f-9230-4692-b812-7b030ed52989 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:57.726865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e3258-7ba6-4238-8780-c1d43826ab2a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:57.728329 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a46e3258-7ba6-4238-8780-c1d43826ab2a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:57.742006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:57.747877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:57.763192 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a5adb-c5eb-40aa-8e01-aa5d415e3204 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:57.764115 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a5adb-c5eb-40aa-8e01-aa5d415e3204 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:57.765122 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a5adb-c5eb-40aa-8e01-aa5d415e3204 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:57.765536 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c99a5adb-c5eb-40aa-8e01-aa5d415e3204 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:57.775962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a5adb-c5eb-40aa-8e01-aa5d415e3204 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:57.777598 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c99a5adb-c5eb-40aa-8e01-aa5d415e3204 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:57.794067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c99a5adb-c5eb-40aa-8e01-aa5d415e3204 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 13:53:57.794067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:57.794067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:57.794067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:57.794067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:57.794067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:57.794067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:57.794067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:57.794067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:57.794067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:57.794067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:57.794067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:57.794067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:53:57.794067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:53:57.794067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:53:57.794067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:53:57.795178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:57.795178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:57.795178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:53:57.795178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:57.795178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:53:57.795178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:53:57.795178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:57.795178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:57.795178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:57.795178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:57.795178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:57.795178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:57.795178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:57.795178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:57.795178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:57.795178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:57.795178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:57.795178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:57.796141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:57.796141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:57.796141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:53:57.796141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:53:57.796141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:53:57.796141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:53:57.796141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:53:57.796141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:53:57.796141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 11 13:53:57.796141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:57.796141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 340/658] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:53:57 2022] GET /identity/v3/users/neutron => generated 85 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:57.796752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a5adb-c5eb-40aa-8e01-aa5d415e3204 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:57.865381 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 319/659] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 464 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:57.866694 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e3258-7ba6-4238-8780-c1d43826ab2a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:57.923863 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c99a5adb-c5eb-40aa-8e01-aa5d415e3204 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:57.965525 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505a908-3f1d-42e7-af4b-9235c025b26b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:57.966339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505a908-3f1d-42e7-af4b-9235c025b26b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:57.966623 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505a908-3f1d-42e7-af4b-9235c025b26b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:57.966897 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e505a908-3f1d-42e7-af4b-9235c025b26b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:57.978112 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505a908-3f1d-42e7-af4b-9235c025b26b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:57.979536 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e505a908-3f1d-42e7-af4b-9235c025b26b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:57.980285 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a46e3258-7ba6-4238-8780-c1d43826ab2a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:57.997456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:57.997615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-e505a908-3f1d-42e7-af4b-9235c025b26b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:57.997615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:57.997615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:57.997615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:57.997615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:57.997615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:57.997615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:57.997615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:57.997615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:57.997615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:57.997615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:57.997615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:57.997615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:57.997615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:57.997615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:57.998494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:57.998494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:57.998494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:57.998494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:57.998494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:57.998494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:57.998494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:57.998494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:57.998494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:57.998494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:57.998494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:57.998494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:57.998494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:57.998494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:57.998494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:57.998494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:57.998494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:57.999879 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:57.999879 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:57.999879 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:57.999879 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:57.999879 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 341/660] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:58.000161 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505a908-3f1d-42e7-af4b-9235c025b26b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:58.024280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2facbe-2612-45dd-8aa3-8b5da1921ed6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:58.024949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2facbe-2612-45dd-8aa3-8b5da1921ed6 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:58.025232 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2facbe-2612-45dd-8aa3-8b5da1921ed6 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:58.025504 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2facbe-2612-45dd-8aa3-8b5da1921ed6 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:58.026361 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2facbe-2612-45dd-8aa3-8b5da1921ed6 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:58.026677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2facbe-2612-45dd-8aa3-8b5da1921ed6 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:58.028108 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2facbe-2612-45dd-8aa3-8b5da1921ed6 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:58.033446 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:58.052049 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 320/661] 10.222.0.49 () {56 vars in 1139 bytes} [Mon Jul 11 13:53:57 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:58.053553 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a436ef-d197-44a2-9f61-1e9dc0fffa6d None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:58.053847 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a436ef-d197-44a2-9f61-1e9dc0fffa6d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:58.054127 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a436ef-d197-44a2-9f61-1e9dc0fffa6d None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:58.058433 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 321/662] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:58.058433 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a436ef-d197-44a2-9f61-1e9dc0fffa6d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:58.127488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e505a908-3f1d-42e7-af4b-9235c025b26b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:58.165160 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f738785-0805-4377-ae3d-bbf1ef1015a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:58.166014 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f738785-0805-4377-ae3d-bbf1ef1015a9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:58.166250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f738785-0805-4377-ae3d-bbf1ef1015a9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:58.166563 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f738785-0805-4377-ae3d-bbf1ef1015a9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:58.167326 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a436ef-d197-44a2-9f61-1e9dc0fffa6d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:58.178801 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f738785-0805-4377-ae3d-bbf1ef1015a9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:58.180539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f738785-0805-4377-ae3d-bbf1ef1015a9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:58.203839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-5f738785-0805-4377-ae3d-bbf1ef1015a9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:58.203839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:58.203839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:58.203839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:58.203839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:58.203839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:58.203839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:58.203839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:58.203839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:58.203839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:58.203839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:58.203839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:58.203839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:58.203839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:58.203839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:58.204782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:58.204782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:58.204782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:58.204782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:58.204782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:58.204782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:58.204782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:58.204782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:58.204782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:58.204782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:58.204782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:58.204782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:58.204782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:58.204782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:58.204782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:58.204782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:58.204782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:58.205650 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:58.205650 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:58.205650 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:58.205650 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:58.205650 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 342/663] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:58.205925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f738785-0805-4377-ae3d-bbf1ef1015a9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:58.209368 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a71f8e-b22b-4db8-973c-ece85ffbd540 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:58.210280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a71f8e-b22b-4db8-973c-ece85ffbd540 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:58.210601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a71f8e-b22b-4db8-973c-ece85ffbd540 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:58.210881 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a71f8e-b22b-4db8-973c-ece85ffbd540 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:58.211623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a71f8e-b22b-4db8-973c-ece85ffbd540 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:58.211934 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a71f8e-b22b-4db8-973c-ece85ffbd540 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:58.213322 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a71f8e-b22b-4db8-973c-ece85ffbd540 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:58.229734 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 322/664] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:58.229734 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9ab33-04e6-4ea7-b4f1-f1681b657070 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:58.229734 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9ab33-04e6-4ea7-b4f1-f1681b657070 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:58.229734 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af9ab33-04e6-4ea7-b4f1-f1681b657070 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:58.245218 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5af9ab33-04e6-4ea7-b4f1-f1681b657070 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:58.246458 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5af9ab33-04e6-4ea7-b4f1-f1681b657070 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:58.276327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:58.302876 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:58.320867 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 323/665] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:58.321896 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9ab33-04e6-4ea7-b4f1-f1681b657070 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:58.335812 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f738785-0805-4377-ae3d-bbf1ef1015a9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:58.376231 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c3dfe-fc0b-49d6-924a-e8e3bcb687bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:58.376899 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c3dfe-fc0b-49d6-924a-e8e3bcb687bc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:58.377185 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c3dfe-fc0b-49d6-924a-e8e3bcb687bc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:58.377467 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a4c3dfe-fc0b-49d6-924a-e8e3bcb687bc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:58.437225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af9ab33-04e6-4ea7-b4f1-f1681b657070 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:58.453943 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c3dfe-fc0b-49d6-924a-e8e3bcb687bc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:58.455479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a4c3dfe-fc0b-49d6-924a-e8e3bcb687bc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:58.473612 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30d99b-82ff-4b87-9593-924fb40f495d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:58.474555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30d99b-82ff-4b87-9593-924fb40f495d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:58.474986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30d99b-82ff-4b87-9593-924fb40f495d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:58.475496 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf30d99b-82ff-4b87-9593-924fb40f495d None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:58.476374 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30d99b-82ff-4b87-9593-924fb40f495d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:58.476857 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30d99b-82ff-4b87-9593-924fb40f495d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:58.478409 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf30d99b-82ff-4b87-9593-924fb40f495d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:58.490871 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 324/666] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:58.491703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30d99b-82ff-4b87-9593-924fb40f495d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:58.521844 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:58.544994 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:58.574026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 343/667] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:53:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 11 13:53:58.574026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c3dfe-fc0b-49d6-924a-e8e3bcb687bc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:58.611625 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf30d99b-82ff-4b87-9593-924fb40f495d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:58.648263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc9d25-9baa-4193-a66c-8f41b0cfbc37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:58.649065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc9d25-9baa-4193-a66c-8f41b0cfbc37 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:58.649347 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc9d25-9baa-4193-a66c-8f41b0cfbc37 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:58.649622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3bc9d25-9baa-4193-a66c-8f41b0cfbc37 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:58.657383 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc9d25-9baa-4193-a66c-8f41b0cfbc37 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:58.658879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3bc9d25-9baa-4193-a66c-8f41b0cfbc37 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:58.674535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-f3bc9d25-9baa-4193-a66c-8f41b0cfbc37 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:58.674535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:58.674535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:58.674535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:58.674535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:58.674535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:58.674535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:58.674535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:58.674535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:58.674535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:58.674535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:58.674535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:58.674535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:53:58.674535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:53:58.674535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:53:58.675583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:53:58.675583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:58.675583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:58.675583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:58.675583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:58.675583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:58.675583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:58.675583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:58.675583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:58.675583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:58.675583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:58.675583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:58.675583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:58.675583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:58.675583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:58.675583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:58.675583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:58.676594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:53:58.676594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:53:58.676594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:53:58.676594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:53:58.676594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:53:58.676594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:53:58.676594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:58.676594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:58.680921 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 325/668] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:58.681891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc9d25-9baa-4193-a66c-8f41b0cfbc37 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:58.687581 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a4c3dfe-fc0b-49d6-924a-e8e3bcb687bc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:58.724952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e0ea1-ceea-4ed2-b0aa-0a252bb8d9e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:58.725767 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e0ea1-ceea-4ed2-b0aa-0a252bb8d9e2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:58.726460 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e0ea1-ceea-4ed2-b0aa-0a252bb8d9e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:58.726460 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299e0ea1-ceea-4ed2-b0aa-0a252bb8d9e2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:58.738813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299e0ea1-ceea-4ed2-b0aa-0a252bb8d9e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:58.746835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-299e0ea1-ceea-4ed2-b0aa-0a252bb8d9e2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:58.746835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:58.746835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:58.746835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:58.746835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:58.746835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:58.746835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:58.746835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:58.746835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:58.746835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:58.746835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:58.746835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:58.746835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:58.746835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:58.746835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:58.746835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:58.747831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:58.747831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:58.747831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:58.747831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:58.747831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:58.747831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:58.747831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:58.747831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:58.747831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:58.747831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:58.747831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:58.747831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:58.747831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:58.747831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:58.747831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:58.747831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:58.748752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:58.748752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:58.748752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:58.748752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:58.748752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:58.748752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:58.748752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:58.748752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:58.748752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:58.748752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:58.748752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:53:58.748752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:58.748752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 344/669] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:53:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:58.749482 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e0ea1-ceea-4ed2-b0aa-0a252bb8d9e2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:58.786428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:58.802254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:58.809421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3bc9d25-9baa-4193-a66c-8f41b0cfbc37 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:58.854466 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cf6ac-edcc-4f09-afa0-dc35733ec10e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:58.855668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cf6ac-edcc-4f09-afa0-dc35733ec10e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:58.856080 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cf6ac-edcc-4f09-afa0-dc35733ec10e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:58.856516 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af3cf6ac-edcc-4f09-afa0-dc35733ec10e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:58.864413 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3cf6ac-edcc-4f09-afa0-dc35733ec10e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:58.865892 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af3cf6ac-edcc-4f09-afa0-dc35733ec10e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:58.874484 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299e0ea1-ceea-4ed2-b0aa-0a252bb8d9e2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:58.879891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-af3cf6ac-edcc-4f09-afa0-dc35733ec10e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:58.879891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:58.879891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:58.879891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:58.879891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:58.879891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:58.879891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:58.879891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:58.879891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:58.879891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:58.879891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:58.879891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:58.879891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:53:58.879891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:53:58.879891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:53:58.880971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:53:58.880971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:58.880971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:58.880971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:58.880971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:58.880971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:58.880971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:58.880971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:58.880971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:58.880971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:58.880971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:58.880971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:58.880971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:58.880971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:58.880971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:58.880971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:58.880971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:58.881877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:53:58.881877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:53:58.881877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:53:58.881877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:53:58.881877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:53:58.881877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:53:58.881877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:58.881877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:58.881877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 326/670] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:58 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:58.881877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cf6ac-edcc-4f09-afa0-dc35733ec10e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:58.916884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00046d3-4274-4f0b-b162-3fd4d70ea135 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:58.917685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00046d3-4274-4f0b-b162-3fd4d70ea135 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:58.917974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00046d3-4274-4f0b-b162-3fd4d70ea135 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:58.918255 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00046d3-4274-4f0b-b162-3fd4d70ea135 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:58.929165 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00046d3-4274-4f0b-b162-3fd4d70ea135 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:58.931046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00046d3-4274-4f0b-b162-3fd4d70ea135 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:58.948928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-e00046d3-4274-4f0b-b162-3fd4d70ea135 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:58.948928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:58.948928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:58.948928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:58.948928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:58.948928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:58.948928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:58.948928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:58.948928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:58.948928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:58.948928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:58.948928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:58.948928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:58.948928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:58.948928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:58.949948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:58.949948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:58.949948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:58.949948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:58.949948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:58.949948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:58.949948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:58.949948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:58.949948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:58.949948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:58.949948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:58.949948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:58.949948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:58.949948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:58.949948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:58.949948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:58.949948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:58.950864 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:58.950864 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:58.950864 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:58.950864 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:58.950864 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 345/671] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:58.950864 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00046d3-4274-4f0b-b162-3fd4d70ea135 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:58.987067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af3cf6ac-edcc-4f09-afa0-dc35733ec10e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:59.021413 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd764f6a-7d79-49b6-bf92-22df8393723f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:59.022078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd764f6a-7d79-49b6-bf92-22df8393723f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:59.022398 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd764f6a-7d79-49b6-bf92-22df8393723f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:59.022684 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd764f6a-7d79-49b6-bf92-22df8393723f None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:59.023490 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd764f6a-7d79-49b6-bf92-22df8393723f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:59.023804 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd764f6a-7d79-49b6-bf92-22df8393723f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:59.025238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd764f6a-7d79-49b6-bf92-22df8393723f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:59.039314 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 327/672] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:53:58 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:59.040651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a8c23-2762-4c40-9682-6b8e83e4c993 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:59.041023 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a8c23-2762-4c40-9682-6b8e83e4c993 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:59.041205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508a8c23-2762-4c40-9682-6b8e83e4c993 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:59.042625 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 328/673] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:53:59.043331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a8c23-2762-4c40-9682-6b8e83e4c993 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:59.049804 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:59.052778 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:59.062281 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00046d3-4274-4f0b-b162-3fd4d70ea135 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:59.100304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cde2f4-df17-4cb5-a4d1-e4426ef21ed3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:59.100917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cde2f4-df17-4cb5-a4d1-e4426ef21ed3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:59.101208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cde2f4-df17-4cb5-a4d1-e4426ef21ed3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:59.101485 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54cde2f4-df17-4cb5-a4d1-e4426ef21ed3 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:59.102250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cde2f4-df17-4cb5-a4d1-e4426ef21ed3 None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:59.102596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cde2f4-df17-4cb5-a4d1-e4426ef21ed3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:59.103965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54cde2f4-df17-4cb5-a4d1-e4426ef21ed3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:59.112343 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 346/674] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:53:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:59.113169 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cde2f4-df17-4cb5-a4d1-e4426ef21ed3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:59.155985 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-508a8c23-2762-4c40-9682-6b8e83e4c993 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:59.191445 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd397419-4b67-4aef-a05a-08d00b0fff44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:59.192103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd397419-4b67-4aef-a05a-08d00b0fff44 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:59.192386 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd397419-4b67-4aef-a05a-08d00b0fff44 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:59.192659 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd397419-4b67-4aef-a05a-08d00b0fff44 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:59.193857 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd397419-4b67-4aef-a05a-08d00b0fff44 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:59.194168 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd397419-4b67-4aef-a05a-08d00b0fff44 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:59.196013 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd397419-4b67-4aef-a05a-08d00b0fff44 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:59.209515 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 329/675] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:53:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:59.210711 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93760db2-b683-4a5d-97ba-b9b0d6f22872 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:59.210998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93760db2-b683-4a5d-97ba-b9b0d6f22872 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:59.211275 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93760db2-b683-4a5d-97ba-b9b0d6f22872 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:59.226133 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54cde2f4-df17-4cb5-a4d1-e4426ef21ed3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:59.227839 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-93760db2-b683-4a5d-97ba-b9b0d6f22872 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:53:59.229021 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-93760db2-b683-4a5d-97ba-b9b0d6f22872 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:53:59.260124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccc6ae-f438-429c-b905-3dc2c7ebe0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:59.260786 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccc6ae-f438-429c-b905-3dc2c7ebe0ba None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:59.261067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccc6ae-f438-429c-b905-3dc2c7ebe0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:59.261345 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ccc6ae-f438-429c-b905-3dc2c7ebe0ba None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:59.262062 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccc6ae-f438-429c-b905-3dc2c7ebe0ba None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:59.262395 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccc6ae-f438-429c-b905-3dc2c7ebe0ba None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:59.263809 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ccc6ae-f438-429c-b905-3dc2c7ebe0ba None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:59.276531 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 347/676] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:59.277368 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccc6ae-f438-429c-b905-3dc2c7ebe0ba None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:59.293378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:59.293617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:59.303319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 330/677] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:53:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:53:59.304149 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93760db2-b683-4a5d-97ba-b9b0d6f22872 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:59.380181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ccc6ae-f438-429c-b905-3dc2c7ebe0ba None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:59.414570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93760db2-b683-4a5d-97ba-b9b0d6f22872 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:59.415410 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dc0a7-dcbd-4dbc-af5c-c5d9293329ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:59.416073 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dc0a7-dcbd-4dbc-af5c-c5d9293329ab None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:59.416359 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dc0a7-dcbd-4dbc-af5c-c5d9293329ab None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:59.416633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17dc0a7-dcbd-4dbc-af5c-c5d9293329ab None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:59.417476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17dc0a7-dcbd-4dbc-af5c-c5d9293329ab None admin] RBAC: Adding query filter params (role.id=1f7cf26664d04e5e9331ead293476442, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=6bb1616f632d4c67828cce3fc41945b7) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:59.417783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17dc0a7-dcbd-4dbc-af5c-c5d9293329ab None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:59.419229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17dc0a7-dcbd-4dbc-af5c-c5d9293329ab None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:59.426072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 348/678] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:53:59 2022] GET /identity/v3/role_assignments?user.id=6bb1616f632d4c67828cce3fc41945b7&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=1f7cf26664d04e5e9331ead293476442 => generated 608 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:59.426967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dc0a7-dcbd-4dbc-af5c-c5d9293329ab None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:59.456673 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c36f75-ac04-4d6f-8797-0bc932d73097 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:59.457508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c36f75-ac04-4d6f-8797-0bc932d73097 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:59.457797 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c36f75-ac04-4d6f-8797-0bc932d73097 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:59.458078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9c36f75-ac04-4d6f-8797-0bc932d73097 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:59.470620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c36f75-ac04-4d6f-8797-0bc932d73097 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:59.472183 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9c36f75-ac04-4d6f-8797-0bc932d73097 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:59.495343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-e9c36f75-ac04-4d6f-8797-0bc932d73097 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:59.495343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:59.495343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:59.495343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:59.495343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:59.495343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:59.495343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:59.495343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:59.495343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:59.495343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:59.495343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:59.495343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:59.495343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:53:59.495343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:53:59.495343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:53:59.496346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:53:59.496346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:59.496346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:59.496346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:59.496346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:59.496346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:59.496346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:59.496346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:59.496346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:59.496346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:59.496346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:59.496346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:59.496346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:59.496346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:59.496346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:59.496346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:59.496346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:59.497290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:53:59.497290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:53:59.497290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:53:59.497290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:59.497290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 331/679] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:53:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:59.497579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c36f75-ac04-4d6f-8797-0bc932d73097 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:59.528218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:59.556681 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17dc0a7-dcbd-4dbc-af5c-c5d9293329ab None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:59.562407 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:59.598630 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570231a-e937-464e-b2ee-c580724f5e18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:59.599301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570231a-e937-464e-b2ee-c580724f5e18 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:59.599592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570231a-e937-464e-b2ee-c580724f5e18 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:59.599872 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2570231a-e937-464e-b2ee-c580724f5e18 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:59.600727 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570231a-e937-464e-b2ee-c580724f5e18 None admin] RBAC: Adding query filter params (role.id=1f7cf26664d04e5e9331ead293476442, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=f80d59fec4994e689e857ba2087d3a23) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:59.601049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570231a-e937-464e-b2ee-c580724f5e18 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:59.603839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2570231a-e937-464e-b2ee-c580724f5e18 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:59.610985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 349/680] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:53:59 2022] GET /identity/v3/role_assignments?user.id=f80d59fec4994e689e857ba2087d3a23&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=1f7cf26664d04e5e9331ead293476442 => generated 608 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:59.612401 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570231a-e937-464e-b2ee-c580724f5e18 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:59.627740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9c36f75-ac04-4d6f-8797-0bc932d73097 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:59.678383 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71faf247-46cf-4380-b717-e50f6efa970b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:59.679627 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faf247-46cf-4380-b717-e50f6efa970b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:59.680243 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faf247-46cf-4380-b717-e50f6efa970b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:59.680589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71faf247-46cf-4380-b717-e50f6efa970b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:59.689098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71faf247-46cf-4380-b717-e50f6efa970b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:59.690665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71faf247-46cf-4380-b717-e50f6efa970b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:59.710132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-71faf247-46cf-4380-b717-e50f6efa970b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:59.710132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:59.710132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:59.710132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:59.710132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:59.710132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:59.710132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:59.710132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:59.710132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:59.710132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:59.710132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:59.710132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:59.710132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:53:59.710132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:53:59.710132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:53:59.711141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:53:59.711141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:59.711141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:59.711141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:59.711141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:59.711141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:59.711141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:59.711141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:59.711141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:59.711141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:59.711141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:59.711141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:59.711141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:59.711141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:59.711141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:59.711141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:59.711141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:59.712015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:53:59.712015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:53:59.712015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:53:59.712015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:53:59.712015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:53:59.712015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:53:59.712015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:53:59.712015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:53:59.712015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 332/681] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:53:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:59.712517 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71faf247-46cf-4380-b717-e50f6efa970b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:59.756293 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2570231a-e937-464e-b2ee-c580724f5e18 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:59.815285 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfa061-ddc7-4df6-b31f-22f0af3bb2a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:59.816162 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfa061-ddc7-4df6-b31f-22f0af3bb2a5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:59.816437 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfa061-ddc7-4df6-b31f-22f0af3bb2a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:59.816718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7cfa061-ddc7-4df6-b31f-22f0af3bb2a5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:59.831197 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7cfa061-ddc7-4df6-b31f-22f0af3bb2a5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:53:59.839134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-f7cfa061-ddc7-4df6-b31f-22f0af3bb2a5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:53:59.839134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:53:59.839134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:53:59.839134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:53:59.839134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:53:59.839134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:53:59.839134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:53:59.839134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:53:59.839134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:53:59.839134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:53:59.839134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:53:59.839134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:53:59.839134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:53:59.839134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:53:59.839134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:53:59.839134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:53:59.840743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:53:59.840743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:53:59.840743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:53:59.840743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:53:59.840743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:53:59.840743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:53:59.840743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:53:59.840743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:53:59.840743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:53:59.840743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:53:59.840743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:53:59.840743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:53:59.840743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:53:59.840743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:53:59.840743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:53:59.840743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:53:59.842244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:53:59.842244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:53:59.842244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:53:59.842244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:53:59.842244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:53:59.842244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:53:59.842244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:53:59.842244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:53:59.842244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:53:59.842244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:53:59.842244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:53:59.842244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:53:59.842244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 350/682] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:53:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:53:59.842244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfa061-ddc7-4df6-b31f-22f0af3bb2a5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:59.845948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:59.882996 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71faf247-46cf-4380-b717-e50f6efa970b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:53:59.894761 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:53:59.939710 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617e7bf-1fba-436d-8bdf-c9d45db3ce56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:53:59.940554 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617e7bf-1fba-436d-8bdf-c9d45db3ce56 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:53:59.940838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617e7bf-1fba-436d-8bdf-c9d45db3ce56 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:53:59.941112 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9617e7bf-1fba-436d-8bdf-c9d45db3ce56 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:53:59.941849 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617e7bf-1fba-436d-8bdf-c9d45db3ce56 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:53:59.942154 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617e7bf-1fba-436d-8bdf-c9d45db3ce56 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:53:59.943603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9617e7bf-1fba-436d-8bdf-c9d45db3ce56 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:53:59.960580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 333/683] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:53:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:53:59.961727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617e7bf-1fba-436d-8bdf-c9d45db3ce56 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:53:59.981423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7cfa061-ddc7-4df6-b31f-22f0af3bb2a5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:00.030245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264c6a8-bf93-4fb4-bc8e-e6f2904faf5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:00.031810 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264c6a8-bf93-4fb4-bc8e-e6f2904faf5f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:00.032638 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264c6a8-bf93-4fb4-bc8e-e6f2904faf5f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:00.033125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f264c6a8-bf93-4fb4-bc8e-e6f2904faf5f None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:00.034758 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264c6a8-bf93-4fb4-bc8e-e6f2904faf5f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:00.035618 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264c6a8-bf93-4fb4-bc8e-e6f2904faf5f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:00.037978 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f264c6a8-bf93-4fb4-bc8e-e6f2904faf5f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:00.066409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 351/684] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:53:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:00.074371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264c6a8-bf93-4fb4-bc8e-e6f2904faf5f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:00.113835 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9617e7bf-1fba-436d-8bdf-c9d45db3ce56 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:00.155540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:00.162581 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d041452-9021-4352-b52c-b23d595de8df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:00.163162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d041452-9021-4352-b52c-b23d595de8df None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:00.163455 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d041452-9021-4352-b52c-b23d595de8df None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:00.163741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d041452-9021-4352-b52c-b23d595de8df None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:00.164560 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d041452-9021-4352-b52c-b23d595de8df None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:00.164876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d041452-9021-4352-b52c-b23d595de8df None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:00.166268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d041452-9021-4352-b52c-b23d595de8df None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:00.182037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 334/685] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:53:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:00.183464 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d041452-9021-4352-b52c-b23d595de8df None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:00.209808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f264c6a8-bf93-4fb4-bc8e-e6f2904faf5f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:00.224682 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:00.254734 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3dcb8-85c0-4696-8423-c65ece3082ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:00.255776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3dcb8-85c0-4696-8423-c65ece3082ac None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:00.256165 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3dcb8-85c0-4696-8423-c65ece3082ac None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:00.256550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a3dcb8-85c0-4696-8423-c65ece3082ac None admin] PATH_INFO: `/v3/users/swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:00.265297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3dcb8-85c0-4696-8423-c65ece3082ac None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:00.267071 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a3dcb8-85c0-4696-8423-c65ece3082ac None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:00.294572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-22a3dcb8-85c0-4696-8423-c65ece3082ac None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 13:54:00.294572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:00.294572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:00.294572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:00.294572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:00.294572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:00.294572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:00.294572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:00.294572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:00.294572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:00.294572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:00.294572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:00.294572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:54:00.294572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:54:00.294572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:54:00.294572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:54:00.296145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:00.296145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:00.296145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:54:00.296145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:00.296145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:54:00.296145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:00.296145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:00.296145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:00.296145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:00.296145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:00.296145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:00.296145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:00.296145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:00.296145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:00.296145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:00.296145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:00.296145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:00.296145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:00.297085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:00.297085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:00.297085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:54:00.297085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:54:00.297085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:54:00.297085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:54:00.297085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:54:00.297085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:54:00.297085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 13:54:00.297085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:00.301222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 352/686] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:54:00 2022] GET /identity/v3/users/swift => generated 83 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:00.302236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3dcb8-85c0-4696-8423-c65ece3082ac None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:00.323472 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d041452-9021-4352-b52c-b23d595de8df None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:00.379268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5364c81-d749-4a44-a932-63c3bee27f43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:00.380487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5364c81-d749-4a44-a932-63c3bee27f43 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:00.380929 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5364c81-d749-4a44-a932-63c3bee27f43 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:00.381387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5364c81-d749-4a44-a932-63c3bee27f43 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:00.383450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5364c81-d749-4a44-a932-63c3bee27f43 None admin] RBAC: Adding query filter params (role.id=1f7cf26664d04e5e9331ead293476442, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=ce1c85ee44ef45d2b631e51112b5ea0a) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:00.384042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5364c81-d749-4a44-a932-63c3bee27f43 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:00.385669 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5364c81-d749-4a44-a932-63c3bee27f43 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:00.401989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 335/687] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:54:00 2022] GET /identity/v3/role_assignments?user.id=ce1c85ee44ef45d2b631e51112b5ea0a&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=1f7cf26664d04e5e9331ead293476442 => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:00.401989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5364c81-d749-4a44-a932-63c3bee27f43 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:00.446476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a3dcb8-85c0-4696-8423-c65ece3082ac None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:00.504487 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:00.536049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb13ec2-158a-4490-8434-65e46ad7f2b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:00.537229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb13ec2-158a-4490-8434-65e46ad7f2b9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:00.537664 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb13ec2-158a-4490-8434-65e46ad7f2b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:00.538088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fb13ec2-158a-4490-8434-65e46ad7f2b9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:00.551518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:00.557967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5364c81-d749-4a44-a932-63c3bee27f43 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:00.569754 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb13ec2-158a-4490-8434-65e46ad7f2b9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:00.572391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fb13ec2-158a-4490-8434-65e46ad7f2b9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:00.598466 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a379612-eabb-4382-95c4-cfa2cb81bc14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:00.599540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a379612-eabb-4382-95c4-cfa2cb81bc14 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:00.600034 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a379612-eabb-4382-95c4-cfa2cb81bc14 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:00.600518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a379612-eabb-4382-95c4-cfa2cb81bc14 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:00.601621 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a379612-eabb-4382-95c4-cfa2cb81bc14 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:00.602145 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a379612-eabb-4382-95c4-cfa2cb81bc14 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:00.603841 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a379612-eabb-4382-95c4-cfa2cb81bc14 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:00.618578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-9fb13ec2-158a-4490-8434-65e46ad7f2b9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:00.618578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:00.618578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:00.618578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:00.618578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:00.618578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:00.618578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:00.618578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:00.618578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:00.618578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:00.618578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:00.618578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:00.618578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:00.618578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:00.618578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:00.619888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:00.619888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:00.619888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:00.619888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:00.619888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:00.619888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:00.619888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:00.619888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:00.619888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:00.619888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:00.619888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:00.619888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:00.619888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:00.619888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:00.619888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:00.619888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:00.619888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:00.620793 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:00.620793 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:00.620793 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:00.620793 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:00.620793 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 353/688] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 318 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:00.625611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb13ec2-158a-4490-8434-65e46ad7f2b9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:00.628665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 336/689] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Jul 11 13:54:00 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:00.652506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a379612-eabb-4382-95c4-cfa2cb81bc14 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:00.758148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fb13ec2-158a-4490-8434-65e46ad7f2b9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:00.804166 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570d19c-22de-430f-94c2-2b6d87737c6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:00.804166 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570d19c-22de-430f-94c2-2b6d87737c6e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:00.804166 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570d19c-22de-430f-94c2-2b6d87737c6e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:00.804166 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1570d19c-22de-430f-94c2-2b6d87737c6e None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:00.804652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570d19c-22de-430f-94c2-2b6d87737c6e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:00.804652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570d19c-22de-430f-94c2-2b6d87737c6e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:00.805244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1570d19c-22de-430f-94c2-2b6d87737c6e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:00.817179 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a379612-eabb-4382-95c4-cfa2cb81bc14 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:00.823686 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 354/690] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:00.829519 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570d19c-22de-430f-94c2-2b6d87737c6e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:00.858081 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bdaef0-be10-45a0-b20c-84c05b507164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:00.859194 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bdaef0-be10-45a0-b20c-84c05b507164 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:00.859690 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bdaef0-be10-45a0-b20c-84c05b507164 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:00.860369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bdaef0-be10-45a0-b20c-84c05b507164 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:00.880212 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:00.884437 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:00.945539 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bdaef0-be10-45a0-b20c-84c05b507164 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:00.947369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bdaef0-be10-45a0-b20c-84c05b507164 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:00.989141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1570d19c-22de-430f-94c2-2b6d87737c6e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:01.037320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f1d3e-9042-4932-bd3a-e27338a455a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:01.038488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f1d3e-9042-4932-bd3a-e27338a455a8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:01.038996 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f1d3e-9042-4932-bd3a-e27338a455a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:01.039499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa2f1d3e-9042-4932-bd3a-e27338a455a8 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:01.048558 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f1d3e-9042-4932-bd3a-e27338a455a8 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:01.050210 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa2f1d3e-9042-4932-bd3a-e27338a455a8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:01.067086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-fa2f1d3e-9042-4932-bd3a-e27338a455a8 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 13:54:01.067086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:01.067086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:01.067086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:01.067086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:01.067086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:01.067086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:01.067086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:01.067086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:01.067086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:01.067086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:01.067086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:01.067086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:54:01.067086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:54:01.067086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:54:01.067086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:54:01.068187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:01.068187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:01.068187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:54:01.068187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:01.068187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:54:01.068187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:01.068187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:01.068187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:01.068187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:01.068187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:01.068187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:01.068187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:01.068187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:01.068187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:01.068187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:01.068187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:01.068187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:01.068187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:01.069119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:01.069119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:01.069119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:54:01.069119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:54:01.069119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:54:01.069119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:54:01.069119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:54:01.069119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:54:01.069119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 13:54:01.069119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:01.069119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 355/691] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Jul 11 13:54:00 2022] GET /identity/v3/users/nova => generated 82 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:01.075848 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 337/692] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:54:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 13:54:01.078956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f1d3e-9042-4932-bd3a-e27338a455a8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:01.080677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bdaef0-be10-45a0-b20c-84c05b507164 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:01.135555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:01.193562 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bdaef0-be10-45a0-b20c-84c05b507164 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:01.203348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:01.210350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa2f1d3e-9042-4932-bd3a-e27338a455a8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:01.233841 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa62f31-5ab7-46c9-a757-2319ce2da676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:01.235056 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa62f31-5ab7-46c9-a757-2319ce2da676 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:01.235542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa62f31-5ab7-46c9-a757-2319ce2da676 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:01.235996 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aa62f31-5ab7-46c9-a757-2319ce2da676 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:01.250112 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa62f31-5ab7-46c9-a757-2319ce2da676 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:01.250338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f8eb6-3792-46ea-966a-15841b8e9d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:01.251151 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f8eb6-3792-46ea-966a-15841b8e9d1d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:01.251656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f8eb6-3792-46ea-966a-15841b8e9d1d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:01.252201 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252f8eb6-3792-46ea-966a-15841b8e9d1d None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:01.252912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aa62f31-5ab7-46c9-a757-2319ce2da676 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:01.253759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f8eb6-3792-46ea-966a-15841b8e9d1d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:01.254168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f8eb6-3792-46ea-966a-15841b8e9d1d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:01.255782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-252f8eb6-3792-46ea-966a-15841b8e9d1d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:01.273185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-8aa62f31-5ab7-46c9-a757-2319ce2da676 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:01.273185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:01.273185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:01.273185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:01.273185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:01.273185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:01.273185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:01.273185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:01.273185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:01.273185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:01.273185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:01.273185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:01.273185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:01.273185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:01.273185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:01.274167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:01.274167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:01.274167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:01.274167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:01.274167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:01.274167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:01.274167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:01.274167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:01.274167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:01.274167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:01.274167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:01.274167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:01.274167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:01.274167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:01.274167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:01.274167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:01.274167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:01.275258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:01.275258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:01.275258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:01.275258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:01.277401 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 338/694] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:01.284474 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 356/694] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Jul 11 13:54:01 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:01.284646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa62f31-5ab7-46c9-a757-2319ce2da676 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:01.284728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f8eb6-3792-46ea-966a-15841b8e9d1d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:01.416938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252f8eb6-3792-46ea-966a-15841b8e9d1d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:01.461514 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba4556-8373-47a7-b434-82760a281b41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:01.462293 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba4556-8373-47a7-b434-82760a281b41 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:01.462721 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba4556-8373-47a7-b434-82760a281b41 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:01.463002 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ba4556-8373-47a7-b434-82760a281b41 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:01.490875 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:01.558500 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba4556-8373-47a7-b434-82760a281b41 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:01.559992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ba4556-8373-47a7-b434-82760a281b41 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:01.572919 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aa62f31-5ab7-46c9-a757-2319ce2da676 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:01.575517 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:01.652120 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cd107-48bc-4e26-900b-229d21fe09a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:01.653273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cd107-48bc-4e26-900b-229d21fe09a2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:01.653772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cd107-48bc-4e26-900b-229d21fe09a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:01.654391 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e3cd107-48bc-4e26-900b-229d21fe09a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:01.655973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3cd107-48bc-4e26-900b-229d21fe09a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:01.656512 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3cd107-48bc-4e26-900b-229d21fe09a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:01.658228 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e3cd107-48bc-4e26-900b-229d21fe09a2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:01.688152 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 339/695] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 405 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:01.693177 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 357/696] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:54:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jul 11 13:54:01.694862 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cd107-48bc-4e26-900b-229d21fe09a2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:01.697978 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba4556-8373-47a7-b434-82760a281b41 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:01.770759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:01.836758 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ba4556-8373-47a7-b434-82760a281b41 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:01.888627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a68103cf-99f2-4514-ab00-8aed3d26cd2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:01.890489 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68103cf-99f2-4514-ab00-8aed3d26cd2a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:01.890939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68103cf-99f2-4514-ab00-8aed3d26cd2a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:01.891378 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a68103cf-99f2-4514-ab00-8aed3d26cd2a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:01.911219 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68103cf-99f2-4514-ab00-8aed3d26cd2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:01.912823 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a68103cf-99f2-4514-ab00-8aed3d26cd2a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:01.942155 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e3cd107-48bc-4e26-900b-229d21fe09a2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:01.949714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a68103cf-99f2-4514-ab00-8aed3d26cd2a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:01.949714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:01.949714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:01.949714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:01.949714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:01.949714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:01.949714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:01.949714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:01.949714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:01.949714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:01.949714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:01.949714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:01.949714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:01.949714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:01.949714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:01.950820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:01.950820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:01.950820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:01.950820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:01.950820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:01.950820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:01.950820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:01.950820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:01.950820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:01.950820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:01.950820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:01.950820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:01.950820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:01.950820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:01.950820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:01.950820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:01.950820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:01.952397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:01.952397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:01.952397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:01.952397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:01.964841 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 358/697] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:01.984145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c63e3c-e180-4740-8ea2-fa137168fe56 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:01.984619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c63e3c-e180-4740-8ea2-fa137168fe56 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:01.985099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76c63e3c-e180-4740-8ea2-fa137168fe56 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:01.988404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 359/698] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:01 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:54:01.993090 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c63e3c-e180-4740-8ea2-fa137168fe56 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:02.027413 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2791fedf-5698-48d2-a01f-c709ac233b07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:02.028724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791fedf-5698-48d2-a01f-c709ac233b07 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:02.029258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791fedf-5698-48d2-a01f-c709ac233b07 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:02.029940 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2791fedf-5698-48d2-a01f-c709ac233b07 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:02.039606 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791fedf-5698-48d2-a01f-c709ac233b07 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:02.041611 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2791fedf-5698-48d2-a01f-c709ac233b07 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:02.058417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-2791fedf-5698-48d2-a01f-c709ac233b07 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:02.058417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:02.058417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:02.058417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:02.058417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:02.058417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:02.058417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:02.058417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:02.058417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:02.058417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:02.058417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:02.058417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:02.058417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:54:02.058417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:54:02.058417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:54:02.059393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:54:02.059393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:02.059393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:02.059393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:02.059393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:02.059393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:02.059393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:02.059393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:02.059393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:02.059393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:02.059393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:02.059393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:02.059393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:02.059393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:02.059393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:02.059393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:02.059393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:02.060257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:54:02.060257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:54:02.060257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:54:02.060257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:54:02.060257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:54:02.060257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:54:02.060257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:02.060257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:02.062922 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 340/699] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 368 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:02.064813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b180be3-b470-4ea0-916d-cbdbf079a125 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:02.065349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b180be3-b470-4ea0-916d-cbdbf079a125 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:02.065823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b180be3-b470-4ea0-916d-cbdbf079a125 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:02.094286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8b180be3-b470-4ea0-916d-cbdbf079a125 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:02.096550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8b180be3-b470-4ea0-916d-cbdbf079a125 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:02.120902 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:02.179393 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76c63e3c-e180-4740-8ea2-fa137168fe56 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:02.183285 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:02.198452 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 341/700] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:54:02.198452 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b180be3-b470-4ea0-916d-cbdbf079a125 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:02.260992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0372e7f-7411-4b97-b740-f8975f074c42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:02.261837 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0372e7f-7411-4b97-b740-f8975f074c42 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:02.262211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0372e7f-7411-4b97-b740-f8975f074c42 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:02.262426 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0372e7f-7411-4b97-b740-f8975f074c42 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:02.263144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0372e7f-7411-4b97-b740-f8975f074c42 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:02.263450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0372e7f-7411-4b97-b740-f8975f074c42 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:02.265481 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0372e7f-7411-4b97-b740-f8975f074c42 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:02.289807 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 360/701] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:02.289807 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596a2b8-db81-4d81-911e-0c1427576036 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:02.289807 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596a2b8-db81-4d81-911e-0c1427576036 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:02.289807 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c596a2b8-db81-4d81-911e-0c1427576036 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:02.289807 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 361/702] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:02 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:02.292300 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596a2b8-db81-4d81-911e-0c1427576036 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:02.340363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b180be3-b470-4ea0-916d-cbdbf079a125 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:02.380351 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca11344c-682d-4ee6-aeeb-0ab1585024ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:02.381024 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11344c-682d-4ee6-aeeb-0ab1585024ac None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:02.381308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11344c-682d-4ee6-aeeb-0ab1585024ac None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:02.381580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca11344c-682d-4ee6-aeeb-0ab1585024ac None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:02.382438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca11344c-682d-4ee6-aeeb-0ab1585024ac None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:02.382739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca11344c-682d-4ee6-aeeb-0ab1585024ac None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:02.384168 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca11344c-682d-4ee6-aeeb-0ab1585024ac None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:02.405770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 342/703] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:54:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:02.405770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebd2cf-8207-4438-a286-b3cf50f8c21c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:02.405770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebd2cf-8207-4438-a286-b3cf50f8c21c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:02.405770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caebd2cf-8207-4438-a286-b3cf50f8c21c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:02.423920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-caebd2cf-8207-4438-a286-b3cf50f8c21c None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:02.425276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-caebd2cf-8207-4438-a286-b3cf50f8c21c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:02.430353 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c596a2b8-db81-4d81-911e-0c1427576036 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:02.434521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:02.470251 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a1299-b91f-48e9-af2b-c32f664bde4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:02.471863 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a1299-b91f-48e9-af2b-c32f664bde4a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:02.472146 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a1299-b91f-48e9-af2b-c32f664bde4a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:02.472420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a1299-b91f-48e9-af2b-c32f664bde4a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:02.473578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a1299-b91f-48e9-af2b-c32f664bde4a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:02.473578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a1299-b91f-48e9-af2b-c32f664bde4a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:02.506261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-fe2a1299-b91f-48e9-af2b-c32f664bde4a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 13:54:02.506261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:02.506261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:02.506261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:02.506261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:02.506261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:02.506261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:02.506261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:02.506261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 13:54:02.506261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:02.506261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:02.506261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:02.506261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:02.506261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:02.506261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:02.506261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:02.508511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:02.508511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:02.508511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:02.508511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:02.508511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:02.508511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:02.508511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:02.508511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:02.508511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:02.508511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:02.508511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:02.508511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:02.508511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:02.508511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:02.508511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:02.508511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:02.510286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:02.510286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:02.510286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:02.510286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:02.510286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:02.510286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:02.510286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:02.510286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:02.510286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:02.510286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:02.510286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:02.510286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:02.510286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:02.510286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 13:54:02.510286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:02.510286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 362/704] 10.222.0.49 () {56 vars in 1129 bytes} [Mon Jul 11 13:54:02 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:02.512444 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a1299-b91f-48e9-af2b-c32f664bde4a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:02.526285 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 343/705] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:54:02.526285 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebd2cf-8207-4438-a286-b3cf50f8c21c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:02.555468 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:02.648460 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a1299-b91f-48e9-af2b-c32f664bde4a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:02.663765 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caebd2cf-8207-4438-a286-b3cf50f8c21c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:02.688340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681bd77-7207-4259-8354-8b679eb087c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:02.689136 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681bd77-7207-4259-8354-8b679eb087c5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:02.689421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681bd77-7207-4259-8354-8b679eb087c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:02.689699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681bd77-7207-4259-8354-8b679eb087c5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:02.696893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681bd77-7207-4259-8354-8b679eb087c5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:02.698657 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681bd77-7207-4259-8354-8b679eb087c5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:02.703089 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862278f-229f-4f57-8ffa-0380fda1f5c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:02.703747 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862278f-229f-4f57-8ffa-0380fda1f5c9 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:02.704030 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862278f-229f-4f57-8ffa-0380fda1f5c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:02.704306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9862278f-229f-4f57-8ffa-0380fda1f5c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:02.705143 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862278f-229f-4f57-8ffa-0380fda1f5c9 None admin] RBAC: Adding query filter params (role.id=1f7cf26664d04e5e9331ead293476442, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=d9c1369033a44c1c81d4c1c9ff2bbedb) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:02.705456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862278f-229f-4f57-8ffa-0380fda1f5c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:02.706932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9862278f-229f-4f57-8ffa-0380fda1f5c9 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:02.714646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-1681bd77-7207-4259-8354-8b679eb087c5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:02.714646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:02.714646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:02.714646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:02.714646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:02.714646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:02.714646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:02.714646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:02.714646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:02.714646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:02.714646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:02.714646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:02.714646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:54:02.714646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:54:02.714646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:54:02.715740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:54:02.715740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:02.715740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:02.715740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:02.715740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:02.715740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:02.715740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:02.715740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:02.715740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:02.715740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:02.715740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:02.715740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:02.715740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:02.715740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:02.715740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:02.715740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:02.715740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:02.716887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:54:02.716887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:54:02.716887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:54:02.716887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:54:02.716887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:54:02.716887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:54:02.716887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:02.716887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:02.719122 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 363/706] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:02.719401 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 344/707] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:54:02 2022] GET /identity/v3/role_assignments?user.id=d9c1369033a44c1c81d4c1c9ff2bbedb&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=1f7cf26664d04e5e9331ead293476442 => generated 608 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:02.730761 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681bd77-7207-4259-8354-8b679eb087c5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:02.730968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862278f-229f-4f57-8ffa-0380fda1f5c9 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:02.739320 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:02.841649 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:02.868137 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681bd77-7207-4259-8354-8b679eb087c5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:02.883977 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9862278f-229f-4f57-8ffa-0380fda1f5c9 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:02.925164 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd055cb-ecc3-4079-8464-16f0ff58bc9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:02.926517 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd055cb-ecc3-4079-8464-16f0ff58bc9b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:02.926820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd055cb-ecc3-4079-8464-16f0ff58bc9b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:02.927108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd055cb-ecc3-4079-8464-16f0ff58bc9b None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:02.928169 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd055cb-ecc3-4079-8464-16f0ff58bc9b None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:02.928488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd055cb-ecc3-4079-8464-16f0ff58bc9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:02.930180 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd055cb-ecc3-4079-8464-16f0ff58bc9b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:02.932642 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42bdd4-bc2e-4fe1-9996-d565bc7716ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:02.933583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42bdd4-bc2e-4fe1-9996-d565bc7716ee None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:02.933963 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42bdd4-bc2e-4fe1-9996-d565bc7716ee None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:02.934391 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a42bdd4-bc2e-4fe1-9996-d565bc7716ee None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:02.936727 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 364/708] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Jul 11 13:54:02 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:02.937763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd055cb-ecc3-4079-8464-16f0ff58bc9b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:02.950815 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42bdd4-bc2e-4fe1-9996-d565bc7716ee None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:02.953467 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a42bdd4-bc2e-4fe1-9996-d565bc7716ee None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:02.978419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-8a42bdd4-bc2e-4fe1-9996-d565bc7716ee None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:02.978419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:02.978419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:02.978419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:02.978419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:02.978419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:02.978419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:02.978419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:02.978419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:02.978419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:02.978419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:02.978419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:02.978419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:02.978419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:02.978419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:02.979559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:02.979559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:02.979559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:02.979559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:02.979559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:02.979559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:02.979559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:02.979559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:02.979559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:02.979559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:02.979559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:02.979559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:02.979559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:02.979559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:02.979559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:02.979559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:02.979559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:02.980670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:02.980670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:02.980670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:02.980670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:02.982518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 345/709] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:02.983355 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42bdd4-bc2e-4fe1-9996-d565bc7716ee None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:03.075908 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:03.122291 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd055cb-ecc3-4079-8464-16f0ff58bc9b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:03.160826 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a42bdd4-bc2e-4fe1-9996-d565bc7716ee None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:03.170603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1b564-cc9f-42cf-9a04-3f35a4de75b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:03.171770 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1b564-cc9f-42cf-9a04-3f35a4de75b6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:03.172215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1b564-cc9f-42cf-9a04-3f35a4de75b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:03.172757 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f1b564-cc9f-42cf-9a04-3f35a4de75b6 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:03.173766 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1b564-cc9f-42cf-9a04-3f35a4de75b6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:03.174286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1b564-cc9f-42cf-9a04-3f35a4de75b6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:03.176696 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f1b564-cc9f-42cf-9a04-3f35a4de75b6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:03.194838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 365/710] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:54:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:03.196020 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1b564-cc9f-42cf-9a04-3f35a4de75b6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:03.219168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:03.219552 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b179f19-6c00-4e98-a104-a0695f7d6556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:03.221109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b179f19-6c00-4e98-a104-a0695f7d6556 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:03.221500 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b179f19-6c00-4e98-a104-a0695f7d6556 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:03.222355 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b179f19-6c00-4e98-a104-a0695f7d6556 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:03.223239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b179f19-6c00-4e98-a104-a0695f7d6556 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:03.223660 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b179f19-6c00-4e98-a104-a0695f7d6556 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:03.225474 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b179f19-6c00-4e98-a104-a0695f7d6556 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:03.233762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 346/711] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Jul 11 13:54:02 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:03.235014 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b179f19-6c00-4e98-a104-a0695f7d6556 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:03.412550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f1b564-cc9f-42cf-9a04-3f35a4de75b6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:03.426114 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b179f19-6c00-4e98-a104-a0695f7d6556 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:03.457022 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:03.461818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb4ef5-6573-4985-8bae-666e7e39c3dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:03.462589 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb4ef5-6573-4985-8bae-666e7e39c3dc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:03.462880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb4ef5-6573-4985-8bae-666e7e39c3dc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:03.463164 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35fb4ef5-6573-4985-8bae-666e7e39c3dc None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:03.463914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb4ef5-6573-4985-8bae-666e7e39c3dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:03.464229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb4ef5-6573-4985-8bae-666e7e39c3dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:03.465619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35fb4ef5-6573-4985-8bae-666e7e39c3dc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:03.474404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429cde01-2c4d-4299-bd9c-a220e1d76493 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:03.475829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429cde01-2c4d-4299-bd9c-a220e1d76493 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:03.476214 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429cde01-2c4d-4299-bd9c-a220e1d76493 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:03.476609 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429cde01-2c4d-4299-bd9c-a220e1d76493 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:03.478009 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429cde01-2c4d-4299-bd9c-a220e1d76493 None admin] RBAC: Adding query filter params (role.id=6964639a7f6c41e8a6701ef533151c41, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=ef5dac3f311e43809a46995bab56e780) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:03.478482 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429cde01-2c4d-4299-bd9c-a220e1d76493 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:03.480385 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429cde01-2c4d-4299-bd9c-a220e1d76493 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:03.481401 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 366/712] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 286 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:03.492615 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 347/713] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:54:03 2022] GET /identity/v3/role_assignments?user.id=ef5dac3f311e43809a46995bab56e780&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 259 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:03.495072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb4ef5-6573-4985-8bae-666e7e39c3dc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:03.516345 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b092e-b1ec-488c-b86d-050d062270db None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:03.516631 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b092e-b1ec-488c-b86d-050d062270db None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:03.516915 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-881b092e-b1ec-488c-b86d-050d062270db None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:03.518718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 348/714] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:03.524554 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebb043-53c3-4e77-b50a-6885a1891203 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:03.524821 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebb043-53c3-4e77-b50a-6885a1891203 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:03.525101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ebb043-53c3-4e77-b50a-6885a1891203 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:03.555807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a2ebb043-53c3-4e77-b50a-6885a1891203 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:03.557349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a2ebb043-53c3-4e77-b50a-6885a1891203 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:03.634037 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:03.674629 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35fb4ef5-6573-4985-8bae-666e7e39c3dc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:03.702257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 349/715] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:03 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:54:03.723046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1de7d-1c7d-4ece-867c-696b508643c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:03.723852 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1de7d-1c7d-4ece-867c-696b508643c0 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:03.724214 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1de7d-1c7d-4ece-867c-696b508643c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:03.724570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce1de7d-1c7d-4ece-867c-696b508643c0 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:03.725388 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1de7d-1c7d-4ece-867c-696b508643c0 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:03.726962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce1de7d-1c7d-4ece-867c-696b508643c0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:03.740106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebb043-53c3-4e77-b50a-6885a1891203 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:03.755409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 367/716] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 11 13:54:03 2022] POST /identity/v3/users => generated 254 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:54:03.921281 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ebb043-53c3-4e77-b50a-6885a1891203 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:03.948176 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:03.999978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d40e86-c4a5-4a82-a353-ceae2c54bc91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:04.000841 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d40e86-c4a5-4a82-a353-ceae2c54bc91 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:04.001129 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d40e86-c4a5-4a82-a353-ceae2c54bc91 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:04.001476 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d40e86-c4a5-4a82-a353-ceae2c54bc91 None admin] PATH_INFO: `/v3/services/network` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:04.002729 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d40e86-c4a5-4a82-a353-ceae2c54bc91 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:04.005135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d40e86-c4a5-4a82-a353-ceae2c54bc91 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:04.030906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-32d40e86-c4a5-4a82-a353-ceae2c54bc91 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 13:54:04.030906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:04.030906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:04.030906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:04.030906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:04.030906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:04.030906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:04.030906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:04.030906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 13:54:04.030906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:04.030906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:04.030906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:04.030906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:04.030906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:04.030906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:04.030906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:04.030906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:04.033069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:04.033069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:04.033069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:04.033069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:04.033069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:04.033069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:04.033069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:04.033069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:04.033069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:04.033069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:04.033069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:04.033069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:04.033069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:04.033069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:04.033069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:04.033069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:04.035097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:04.035097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:04.035097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:04.035097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:04.035097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:04.035097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:04.035097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:04.035097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:04.035097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:04.035097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:04.035097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:04.035097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:04.035097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 13:54:04.035097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:04.035097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 350/717] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:03 2022] GET /identity/v3/services/network => generated 88 bytes in 294 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:04.041947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1de7d-1c7d-4ece-867c-696b508643c0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:04.235496 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce1de7d-1c7d-4ece-867c-696b508643c0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:04.268002 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:04.281420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2265a607-5238-4191-8411-c8216119c412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:04.282033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265a607-5238-4191-8411-c8216119c412 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:04.282334 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265a607-5238-4191-8411-c8216119c412 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:04.282626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2265a607-5238-4191-8411-c8216119c412 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:04.283367 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265a607-5238-4191-8411-c8216119c412 None admin] RBAC: Adding query filter params (name=network) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:04.283680 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265a607-5238-4191-8411-c8216119c412 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:04.285149 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2265a607-5238-4191-8411-c8216119c412 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:04.290657 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 368/718] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 11 13:54:04 2022] GET /identity/v3/services?name=network => generated 124 bytes in 250 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:04.296208 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d40e86-c4a5-4a82-a353-ceae2c54bc91 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:04.428907 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d40e86-c4a5-4a82-a353-ceae2c54bc91 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:04.471179 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-592d271c-c2f3-4dec-beed-ba99a3dc3c85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:04.471863 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d271c-c2f3-4dec-beed-ba99a3dc3c85 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:04.472149 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d271c-c2f3-4dec-beed-ba99a3dc3c85 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:04.472429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-592d271c-c2f3-4dec-beed-ba99a3dc3c85 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:04.473164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d271c-c2f3-4dec-beed-ba99a3dc3c85 None admin] RBAC: Adding query filter params (type=network) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:04.473476 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d271c-c2f3-4dec-beed-ba99a3dc3c85 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:04.475721 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-592d271c-c2f3-4dec-beed-ba99a3dc3c85 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:04.486455 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 351/719] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 11 13:54:04 2022] GET /identity/v3/services?type=network => generated 124 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:05.480999 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558ecf2-e751-46cf-a361-beafd964361b None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:05.482179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558ecf2-e751-46cf-a361-beafd964361b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:05.482765 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0558ecf2-e751-46cf-a361-beafd964361b None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:05.484875 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 369/720] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:05.491190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e72591-5293-43d5-be2c-80a9cef226e5 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:05.492148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e72591-5293-43d5-be2c-80a9cef226e5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:05.492653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e72591-5293-43d5-be2c-80a9cef226e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:05.511108 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e4e72591-5293-43d5-be2c-80a9cef226e5 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:05.512660 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e4e72591-5293-43d5-be2c-80a9cef226e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:05.561480 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:05.605043 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 352/721] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:54:05.640622 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558ecf2-e751-46cf-a361-beafd964361b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:05.777480 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0558ecf2-e751-46cf-a361-beafd964361b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:05.831629 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c583120e-d12d-424e-835b-3baa3b8a5421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:05.832440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583120e-d12d-424e-835b-3baa3b8a5421 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:05.832730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583120e-d12d-424e-835b-3baa3b8a5421 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:05.833011 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c583120e-d12d-424e-835b-3baa3b8a5421 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:05.844251 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c583120e-d12d-424e-835b-3baa3b8a5421 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:54:05.851958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c583120e-d12d-424e-835b-3baa3b8a5421 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:54:05.851958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:05.851958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:05.851958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:05.851958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:05.851958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:05.851958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:05.851958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:05.851958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:05.851958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:05.851958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:05.851958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:05.851958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:54:05.851958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:54:05.851958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:54:05.851958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:54:05.852925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:54:05.852925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:54:05.852925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:54:05.852925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:54:05.852925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:05.852925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:05.852925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:05.852925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:05.852925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:05.852925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:05.852925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:05.852925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:05.852925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:05.852925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:05.852925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:05.852925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:05.853759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:05.853759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:05.853759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:05.853759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:05.853759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:54:05.853759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:54:05.853759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:54:05.853759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:54:05.853759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:54:05.853759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:54:05.853759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:54:05.853759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:05.853759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 370/722] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:54:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:05.859814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e72591-5293-43d5-be2c-80a9cef226e5 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:05.921796 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4105a5-de67-4486-aff8-6e282a6778b8 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:05.922130 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4105a5-de67-4486-aff8-6e282a6778b8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:05.922390 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4105a5-de67-4486-aff8-6e282a6778b8 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:05.924059 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 371/723] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:05.929440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439560a-f855-4de3-8bbd-6c60882de0bb None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:05.929731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439560a-f855-4de3-8bbd-6c60882de0bb None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:05.930267 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9439560a-f855-4de3-8bbd-6c60882de0bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:05.949125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9439560a-f855-4de3-8bbd-6c60882de0bb None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:05.950499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9439560a-f855-4de3-8bbd-6c60882de0bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:06.023648 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:06.030860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e72591-5293-43d5-be2c-80a9cef226e5 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:06.078826 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 372/724] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:54:06.079650 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05e0ec-6ed2-4bdb-a48e-e00c43007d1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:06.080495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05e0ec-6ed2-4bdb-a48e-e00c43007d1c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:06.080914 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05e0ec-6ed2-4bdb-a48e-e00c43007d1c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:06.081348 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad05e0ec-6ed2-4bdb-a48e-e00c43007d1c None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:06.082248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05e0ec-6ed2-4bdb-a48e-e00c43007d1c None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:06.082732 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05e0ec-6ed2-4bdb-a48e-e00c43007d1c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:06.084362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad05e0ec-6ed2-4bdb-a48e-e00c43007d1c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:06.096077 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3eed28-eced-4889-8c8c-5bc13cf120f3 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:06.096653 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3eed28-eced-4889-8c8c-5bc13cf120f3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:06.097091 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3eed28-eced-4889-8c8c-5bc13cf120f3 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:06.097611 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 353/725] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:54:05 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:06.099100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 373/726] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:06 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:06.104112 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05e0ec-6ed2-4bdb-a48e-e00c43007d1c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:06.106327 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90097520-95ee-44b4-9170-e75d66914523 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:06.106830 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90097520-95ee-44b4-9170-e75d66914523 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:06.107294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90097520-95ee-44b4-9170-e75d66914523 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:06.131955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-90097520-95ee-44b4-9170-e75d66914523 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:06.133334 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-90097520-95ee-44b4-9170-e75d66914523 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:06.146072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:06.219784 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad05e0ec-6ed2-4bdb-a48e-e00c43007d1c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:06.220829 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 374/727] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 11 13:54:06.221827 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90097520-95ee-44b4-9170-e75d66914523 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:06.282773 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6119f0b-044e-4248-9cd9-0308f9701ad2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:06.283514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6119f0b-044e-4248-9cd9-0308f9701ad2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:06.283801 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6119f0b-044e-4248-9cd9-0308f9701ad2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:06.284079 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6119f0b-044e-4248-9cd9-0308f9701ad2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:06.297176 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6119f0b-044e-4248-9cd9-0308f9701ad2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:06.298771 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6119f0b-044e-4248-9cd9-0308f9701ad2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:06.318050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-c6119f0b-044e-4248-9cd9-0308f9701ad2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:06.318050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:06.318050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:06.318050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:06.318050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:06.318050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:06.318050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:06.318050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:06.318050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:06.318050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:06.318050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:06.318050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:06.318050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:06.318050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:06.318050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:06.319058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:06.319058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:06.319058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:06.319058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:06.319058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:06.319058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:06.319058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:06.319058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:06.319058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:06.319058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:06.319058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:06.319058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:06.319058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:06.319058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:06.319058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:06.319058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:06.319058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:06.319957 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:06.319957 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:06.319957 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:06.319957 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:06.320605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 354/728] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:06.321950 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6119f0b-044e-4248-9cd9-0308f9701ad2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:06.354801 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90097520-95ee-44b4-9170-e75d66914523 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:06.370643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:06.397290 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad83add-4e2d-4eb2-8b99-880fb263e905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:06.398222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad83add-4e2d-4eb2-8b99-880fb263e905 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:06.398542 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad83add-4e2d-4eb2-8b99-880fb263e905 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:06.398824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ad83add-4e2d-4eb2-8b99-880fb263e905 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:06.411330 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ad83add-4e2d-4eb2-8b99-880fb263e905 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:54:06.417021 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:06.422592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-8ad83add-4e2d-4eb2-8b99-880fb263e905 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:54:06.422592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:06.422592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:06.422592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:06.422592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:06.422592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:06.422592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:06.422592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:06.422592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:06.422592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:06.422592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:06.422592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:06.422592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:54:06.422592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:54:06.422592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:54:06.422592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:54:06.424828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:54:06.424828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:54:06.424828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:54:06.424828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:54:06.424828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:06.424828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:06.424828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:06.424828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:06.424828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:06.424828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:06.424828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:06.424828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:06.424828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:06.424828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:06.424828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:06.424828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:06.426861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:06.426861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:06.426861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:06.426861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:06.426861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:54:06.426861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:54:06.426861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:54:06.426861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:54:06.426861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:54:06.426861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:54:06.426861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:54:06.426861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:06.434350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 375/729] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:54:06 2022] GET /identity/v3/roles/admin => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:06.434350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad83add-4e2d-4eb2-8b99-880fb263e905 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:06.434781 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6119f0b-044e-4248-9cd9-0308f9701ad2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:06.469792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1274c2-019f-4a70-9b8d-9297fb58a566 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:06.470598 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1274c2-019f-4a70-9b8d-9297fb58a566 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:06.470891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1274c2-019f-4a70-9b8d-9297fb58a566 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:06.471194 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1274c2-019f-4a70-9b8d-9297fb58a566 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:06.471971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1274c2-019f-4a70-9b8d-9297fb58a566 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:06.473370 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1274c2-019f-4a70-9b8d-9297fb58a566 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:06.500133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 355/730] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 11 13:54:06 2022] POST /identity/v3/services => generated 251 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:54:06.503363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1274c2-019f-4a70-9b8d-9297fb58a566 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:06.603155 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ad83add-4e2d-4eb2-8b99-880fb263e905 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:06.703052 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca06f34-ce9c-4513-8d68-78b32e7178a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:06.706490 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca06f34-ce9c-4513-8d68-78b32e7178a2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:06.706797 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca06f34-ce9c-4513-8d68-78b32e7178a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:06.707157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca06f34-ce9c-4513-8d68-78b32e7178a2 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:06.707954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca06f34-ce9c-4513-8d68-78b32e7178a2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:06.708264 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca06f34-ce9c-4513-8d68-78b32e7178a2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:06.709697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ca06f34-ce9c-4513-8d68-78b32e7178a2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:06.730683 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 376/731] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:06.738083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca06f34-ce9c-4513-8d68-78b32e7178a2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:06.741192 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1274c2-019f-4a70-9b8d-9297fb58a566 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:06.776839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:06.802630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c4695-18c4-4b2b-b481-3fbd390b2471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:06.803348 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c4695-18c4-4b2b-b481-3fbd390b2471 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:06.803608 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c4695-18c4-4b2b-b481-3fbd390b2471 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:06.803888 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d2c4695-18c4-4b2b-b481-3fbd390b2471 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:06.804654 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c4695-18c4-4b2b-b481-3fbd390b2471 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:06.805222 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c4695-18c4-4b2b-b481-3fbd390b2471 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:06.806645 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d2c4695-18c4-4b2b-b481-3fbd390b2471 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:06.818345 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 356/732] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:54:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 318 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:06.827460 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c4695-18c4-4b2b-b481-3fbd390b2471 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:06.843895 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:06.892968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ca06f34-ce9c-4513-8d68-78b32e7178a2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:06.949305 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a68be0-045b-40da-9093-f17cd28b40e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:06.950660 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a68be0-045b-40da-9093-f17cd28b40e8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:06.951192 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a68be0-045b-40da-9093-f17cd28b40e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:06.951736 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a68be0-045b-40da-9093-f17cd28b40e8 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:06.960464 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a68be0-045b-40da-9093-f17cd28b40e8 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:06.962659 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a68be0-045b-40da-9093-f17cd28b40e8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:06.963881 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d2c4695-18c4-4b2b-b481-3fbd390b2471 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:07.001970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-d4a68be0-045b-40da-9093-f17cd28b40e8 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 13:54:07.001970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:07.001970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:07.001970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:07.001970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:07.001970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:07.001970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:07.001970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:07.001970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:07.001970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:07.001970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:07.001970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:07.001970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:54:07.001970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:54:07.001970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:54:07.001970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:54:07.003441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:07.003441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:07.003441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:54:07.003441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:07.003441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:54:07.003441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:07.003441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:07.003441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:07.003441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:07.003441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:07.003441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:07.003441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:07.003441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:07.003441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:07.003441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:07.003441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:07.003441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:07.003441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:07.004534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:07.004534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:07.004534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:54:07.004534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:54:07.004534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:54:07.004534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:54:07.004534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:54:07.004534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:54:07.004534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 13:54:07.004534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:07.012769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 377/733] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:54:06 2022] GET /identity/v3/users/swift => generated 83 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:07.014651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acc58826-65fe-41c2-b938-69d00c906a0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:07.014912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f065b-a42f-488b-8240-234069b55579 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:07.015434 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f065b-a42f-488b-8240-234069b55579 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:07.015563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc58826-65fe-41c2-b938-69d00c906a0c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:07.015725 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc58826-65fe-41c2-b938-69d00c906a0c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:07.016002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acc58826-65fe-41c2-b938-69d00c906a0c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:07.016400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8f065b-a42f-488b-8240-234069b55579 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:07.018208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 378/734] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:07.019181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f065b-a42f-488b-8240-234069b55579 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:07.029306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc58826-65fe-41c2-b938-69d00c906a0c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:07.031436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acc58826-65fe-41c2-b938-69d00c906a0c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:07.052649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-acc58826-65fe-41c2-b938-69d00c906a0c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:07.052649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:07.052649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:07.052649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:07.052649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:07.052649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:07.052649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:07.052649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:07.052649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:07.052649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:07.052649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:07.052649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:07.052649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:07.052649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:07.052649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:07.053620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:07.053620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:07.053620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:07.053620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:07.053620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:07.053620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:07.053620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:07.053620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:07.053620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:07.053620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:07.053620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:07.053620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:07.053620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:07.053620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:07.053620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:07.053620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:07.053620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:07.054517 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:07.054517 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:07.054517 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:07.054517 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:07.057135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 357/735] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:07.058937 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753b115-4bea-4b82-81ed-9b9142ad8656 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:07.059231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753b115-4bea-4b82-81ed-9b9142ad8656 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:07.059511 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9753b115-4bea-4b82-81ed-9b9142ad8656 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:07.081268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9753b115-4bea-4b82-81ed-9b9142ad8656 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:07.082522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9753b115-4bea-4b82-81ed-9b9142ad8656 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:07.135129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:07.147281 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:07.150653 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8f065b-a42f-488b-8240-234069b55579 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:07.159964 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 358/736] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:07 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:54:07.161790 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f61c42-774c-42a0-85b7-303acb45aaab None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:07.162186 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f61c42-774c-42a0-85b7-303acb45aaab None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:07.162589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88f61c42-774c-42a0-85b7-303acb45aaab None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:07.168332 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 359/737] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:07.168332 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f61c42-774c-42a0-85b7-303acb45aaab None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:07.192882 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00e8582-88dd-4fba-8be0-f63854a414a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:07.193941 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e8582-88dd-4fba-8be0-f63854a414a5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:07.194712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e8582-88dd-4fba-8be0-f63854a414a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:07.194712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00e8582-88dd-4fba-8be0-f63854a414a5 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:07.195519 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e8582-88dd-4fba-8be0-f63854a414a5 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:07.195887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e8582-88dd-4fba-8be0-f63854a414a5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:07.197514 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e00e8582-88dd-4fba-8be0-f63854a414a5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:07.221474 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 379/738] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Jul 11 13:54:07 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:07.223543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a882c9-f667-4e54-ab96-cd97b4727342 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:07.224055 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a882c9-f667-4e54-ab96-cd97b4727342 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:07.224535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46a882c9-f667-4e54-ab96-cd97b4727342 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:07.251600 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-46a882c9-f667-4e54-ab96-cd97b4727342 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:07.253678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-46a882c9-f667-4e54-ab96-cd97b4727342 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:07.306332 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88f61c42-774c-42a0-85b7-303acb45aaab None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:07.341426 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 380/739] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:07 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 11 13:54:07.342894 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a882c9-f667-4e54-ab96-cd97b4727342 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:07.364672 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b84e8e-1c48-43d5-8e29-8f825994b25d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:07.365605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b84e8e-1c48-43d5-8e29-8f825994b25d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:07.366494 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b84e8e-1c48-43d5-8e29-8f825994b25d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:07.366786 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b84e8e-1c48-43d5-8e29-8f825994b25d None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:07.367825 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b84e8e-1c48-43d5-8e29-8f825994b25d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:07.368136 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b84e8e-1c48-43d5-8e29-8f825994b25d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:07.370242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b84e8e-1c48-43d5-8e29-8f825994b25d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:07.384214 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 360/740] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:07.385074 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b84e8e-1c48-43d5-8e29-8f825994b25d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:07.431310 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:07.445648 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:07.459762 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46a882c9-f667-4e54-ab96-cd97b4727342 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:07.495105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b84e8e-1c48-43d5-8e29-8f825994b25d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:07.496842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7e55b-b844-4eb2-a8fb-6a4a8c910369 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:07.498776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7e55b-b844-4eb2-a8fb-6a4a8c910369 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:07.498776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7e55b-b844-4eb2-a8fb-6a4a8c910369 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:07.498776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2db7e55b-b844-4eb2-a8fb-6a4a8c910369 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:07.499697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7e55b-b844-4eb2-a8fb-6a4a8c910369 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:07.501059 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2db7e55b-b844-4eb2-a8fb-6a4a8c910369 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:07.518246 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 381/741] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 11 13:54:07 2022] POST /identity/v3/services => generated 240 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:54:07.519118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7e55b-b844-4eb2-a8fb-6a4a8c910369 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:07.550622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5c048-6f9d-4e23-987d-960a3a70feca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:07.551342 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5c048-6f9d-4e23-987d-960a3a70feca None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:07.551576 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5c048-6f9d-4e23-987d-960a3a70feca None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:07.551848 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a5c048-6f9d-4e23-987d-960a3a70feca None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:07.635062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5c048-6f9d-4e23-987d-960a3a70feca None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:07.636562 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70a5c048-6f9d-4e23-987d-960a3a70feca None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:07.670412 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2db7e55b-b844-4eb2-a8fb-6a4a8c910369 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:07.702475 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:07.720952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2c82e-2a5e-42f2-8b2e-4aa3a2726dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:07.721763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2c82e-2a5e-42f2-8b2e-4aa3a2726dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:07.722049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2c82e-2a5e-42f2-8b2e-4aa3a2726dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:07.723103 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a2c82e-2a5e-42f2-8b2e-4aa3a2726dd9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:07.726423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:07.739667 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a2c82e-2a5e-42f2-8b2e-4aa3a2726dd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:54:07.750914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a9a2c82e-2a5e-42f2-8b2e-4aa3a2726dd9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:54:07.750914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:07.750914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:07.750914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:07.750914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:07.750914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:07.750914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:07.750914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:07.750914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:07.750914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:07.750914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:07.750914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:07.750914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:54:07.750914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:54:07.750914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:54:07.750914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:54:07.751898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:54:07.751898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:54:07.751898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:54:07.751898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:54:07.751898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:07.751898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:07.751898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:07.751898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:07.751898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:07.751898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:07.751898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:07.751898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:07.751898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:07.751898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:07.751898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:07.751898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:07.752760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:07.752760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:07.752760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:07.752760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:07.752760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:54:07.752760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:54:07.752760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:54:07.752760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:54:07.752760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:54:07.752760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:54:07.752760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:54:07.752760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:07.755791 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 382/742] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:54:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:07.756919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2c82e-2a5e-42f2-8b2e-4aa3a2726dd9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:07.772334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 361/743] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:54:07 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 387 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 13:54:07.773674 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5c048-6f9d-4e23-987d-960a3a70feca None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:07.898995 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a2c82e-2a5e-42f2-8b2e-4aa3a2726dd9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:07.909944 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70a5c048-6f9d-4e23-987d-960a3a70feca None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:07.951047 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c4af7-8c69-4e92-8999-f9158d1100c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:07.952442 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c4af7-8c69-4e92-8999-f9158d1100c9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:07.952927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c4af7-8c69-4e92-8999-f9158d1100c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:07.953163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09c4af7-8c69-4e92-8999-f9158d1100c9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:07.957748 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e14bd-da7c-41fc-90b2-3f8475e89149 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:07.958569 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e14bd-da7c-41fc-90b2-3f8475e89149 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:07.958934 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e14bd-da7c-41fc-90b2-3f8475e89149 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:07.959286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87e14bd-da7c-41fc-90b2-3f8475e89149 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:07.960627 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e14bd-da7c-41fc-90b2-3f8475e89149 None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:07.961012 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e14bd-da7c-41fc-90b2-3f8475e89149 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:07.961329 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c4af7-8c69-4e92-8999-f9158d1100c9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:07.962680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d87e14bd-da7c-41fc-90b2-3f8475e89149 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:07.962811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09c4af7-8c69-4e92-8999-f9158d1100c9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:07.977650 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 362/744] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:54:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:07.978552 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e14bd-da7c-41fc-90b2-3f8475e89149 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:07.986577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-d09c4af7-8c69-4e92-8999-f9158d1100c9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 13:54:07.986577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:07.986577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:07.986577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:07.986577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:07.986577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:07.986577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:07.986577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:07.986577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:07.986577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:07.986577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:07.986577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:07.986577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:54:07.986577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:54:07.986577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:54:07.986577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:54:07.987602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:07.987602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:07.987602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:54:07.987602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:07.987602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:54:07.987602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:07.987602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:07.987602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:07.987602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:07.987602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:07.987602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:07.987602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:07.987602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:07.987602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:07.987602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:07.987602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:07.987602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:07.987602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:07.989244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:07.989244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:07.989244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:54:07.989244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:54:07.989244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:54:07.989244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:54:07.989244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:54:07.989244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:54:07.989244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 13:54:07.989244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:07.989244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 383/745] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Jul 11 13:54:07 2022] GET /identity/v3/users/nova => generated 82 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:07.989835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c4af7-8c69-4e92-8999-f9158d1100c9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:07.997125 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:08.067405 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:08.095082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d87e14bd-da7c-41fc-90b2-3f8475e89149 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:08.124377 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09c4af7-8c69-4e92-8999-f9158d1100c9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:08.132246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f294ba-6be0-4975-a5bb-24366473838c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:08.133049 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f294ba-6be0-4975-a5bb-24366473838c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:08.133335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f294ba-6be0-4975-a5bb-24366473838c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:08.133620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f294ba-6be0-4975-a5bb-24366473838c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:08.145181 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f294ba-6be0-4975-a5bb-24366473838c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:08.146697 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f294ba-6be0-4975-a5bb-24366473838c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:08.163823 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b04afd-97dc-448e-bc1c-375d12d24ed8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:08.164868 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b04afd-97dc-448e-bc1c-375d12d24ed8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:08.165107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-b8f294ba-6be0-4975-a5bb-24366473838c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:08.165107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:08.165107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:08.165107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:08.165107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:08.165107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:08.165107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:08.165107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:08.165107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:08.165107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:08.165107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:08.165107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:08.165107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:08.165107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:08.165107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:08.166162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:08.166162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:08.166162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:08.166162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:08.166162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:08.166162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:08.166162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:08.166162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:08.166162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:08.166162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:08.166162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:08.166162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:08.166162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:08.166162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:08.166162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:08.166162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:08.166162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:08.167428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:08.167428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:08.167428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:08.167428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:08.167948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b04afd-97dc-448e-bc1c-375d12d24ed8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:08.168359 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b04afd-97dc-448e-bc1c-375d12d24ed8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:08.171106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 363/746] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:08.172243 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f294ba-6be0-4975-a5bb-24366473838c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:08.182018 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b04afd-97dc-448e-bc1c-375d12d24ed8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:08.183881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88b04afd-97dc-448e-bc1c-375d12d24ed8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:08.204355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-88b04afd-97dc-448e-bc1c-375d12d24ed8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:08.204355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:08.204355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:08.204355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:08.204355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:08.204355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:08.204355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:08.204355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:08.204355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:08.204355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:08.204355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:08.204355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:08.204355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:08.204355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:08.204355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:08.205479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:08.205479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:08.205479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:08.205479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:08.205479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:08.205479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:08.205479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:08.205479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:08.205479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:08.205479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:08.205479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:08.205479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:08.205479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:08.205479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:08.205479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:08.205479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:08.205479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:08.206567 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:08.206567 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:08.206567 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:08.206567 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:08.208511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 384/747] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:08.209664 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b04afd-97dc-448e-bc1c-375d12d24ed8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:08.238452 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:08.282170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f294ba-6be0-4975-a5bb-24366473838c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:08.325964 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac820ba-a972-4a44-b1ca-279565f807ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:08.325964 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac820ba-a972-4a44-b1ca-279565f807ff None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:08.325964 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac820ba-a972-4a44-b1ca-279565f807ff None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:08.325964 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac820ba-a972-4a44-b1ca-279565f807ff None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:08.328064 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac820ba-a972-4a44-b1ca-279565f807ff None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:08.328064 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac820ba-a972-4a44-b1ca-279565f807ff None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:08.328064 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac820ba-a972-4a44-b1ca-279565f807ff None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:08.347002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 364/748] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Jul 11 13:54:08 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:08.348129 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac820ba-a972-4a44-b1ca-279565f807ff None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:08.356109 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b04afd-97dc-448e-bc1c-375d12d24ed8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:08.470601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac820ba-a972-4a44-b1ca-279565f807ff None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:08.493564 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:08.498273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:08.520463 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e205d5d-2abf-4d19-9adc-e081c9f77eb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:08.521177 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e205d5d-2abf-4d19-9adc-e081c9f77eb5 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:08.521466 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e205d5d-2abf-4d19-9adc-e081c9f77eb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:08.521738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e205d5d-2abf-4d19-9adc-e081c9f77eb5 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:08.522527 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e205d5d-2abf-4d19-9adc-e081c9f77eb5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:08.522845 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e205d5d-2abf-4d19-9adc-e081c9f77eb5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:08.523489 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12d777-4a29-473a-9b88-15bb3bbd3ea0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:08.524276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e205d5d-2abf-4d19-9adc-e081c9f77eb5 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:08.524555 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12d777-4a29-473a-9b88-15bb3bbd3ea0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:08.525032 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12d777-4a29-473a-9b88-15bb3bbd3ea0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:08.525481 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b12d777-4a29-473a-9b88-15bb3bbd3ea0 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:08.526426 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12d777-4a29-473a-9b88-15bb3bbd3ea0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:08.526915 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12d777-4a29-473a-9b88-15bb3bbd3ea0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:08.528468 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b12d777-4a29-473a-9b88-15bb3bbd3ea0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:08.539992 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 365/749] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:08.541188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e205d5d-2abf-4d19-9adc-e081c9f77eb5 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:08.543105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 385/750] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:08.545972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12d777-4a29-473a-9b88-15bb3bbd3ea0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:08.664958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b12d777-4a29-473a-9b88-15bb3bbd3ea0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:08.680639 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e205d5d-2abf-4d19-9adc-e081c9f77eb5 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:08.704574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b2c0c-55a0-4e25-b2f2-ac391df778b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:08.705392 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b2c0c-55a0-4e25-b2f2-ac391df778b0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:08.705680 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b2c0c-55a0-4e25-b2f2-ac391df778b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:08.705958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655b2c0c-55a0-4e25-b2f2-ac391df778b0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:08.714886 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b2c0c-55a0-4e25-b2f2-ac391df778b0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:08.716407 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655b2c0c-55a0-4e25-b2f2-ac391df778b0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:08.723790 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45a8d1-b889-4c11-95b3-5bb091de7f67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:08.724917 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45a8d1-b889-4c11-95b3-5bb091de7f67 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:08.725388 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45a8d1-b889-4c11-95b3-5bb091de7f67 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:08.725867 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed45a8d1-b889-4c11-95b3-5bb091de7f67 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:08.733211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-655b2c0c-55a0-4e25-b2f2-ac391df778b0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 13:54:08.733211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:08.733211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:08.733211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:08.733211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:08.733211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:08.733211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:08.733211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:08.733211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:08.733211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:08.733211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:08.733211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:08.733211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:54:08.733211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:54:08.733211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:54:08.733211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:54:08.734322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:08.734322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:08.734322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:54:08.734322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:08.734322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:54:08.734322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:08.734322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:08.734322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:08.734322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:08.734322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:08.734322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:08.734322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:08.734322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:08.734322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:08.734322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:08.734322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:08.734322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:08.734322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:08.735358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:08.735358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:08.735358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:54:08.735358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:54:08.735358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:54:08.735358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:54:08.735358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:54:08.735358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:54:08.735358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 13:54:08.735358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:08.735358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 386/751] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 11 13:54:08 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:08.736174 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b2c0c-55a0-4e25-b2f2-ac391df778b0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:08.742879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed45a8d1-b889-4c11-95b3-5bb091de7f67 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:08.744587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed45a8d1-b889-4c11-95b3-5bb091de7f67 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:08.764525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-ed45a8d1-b889-4c11-95b3-5bb091de7f67 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:08.764525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:08.764525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:08.764525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:08.764525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:08.764525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:08.764525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:08.764525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:08.764525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:08.764525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:08.764525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:08.764525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:08.764525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:08.764525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:08.764525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:08.765668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:08.765668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:08.765668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:08.765668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:08.765668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:08.765668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:08.765668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:08.765668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:08.765668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:08.765668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:08.765668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:08.765668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:08.765668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:08.765668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:08.765668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:08.765668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:08.765668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:08.766877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:08.766877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:08.766877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:08.766877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:08.767448 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:08.770273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 366/752] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:08.771290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45a8d1-b889-4c11-95b3-5bb091de7f67 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:08.805537 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:08.851347 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655b2c0c-55a0-4e25-b2f2-ac391df778b0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:08.879978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed45a8d1-b889-4c11-95b3-5bb091de7f67 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:08.887631 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a40d64-5055-4b4c-940b-35542919c646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:08.889338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a40d64-5055-4b4c-940b-35542919c646 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:08.889338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a40d64-5055-4b4c-940b-35542919c646 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:08.889537 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a40d64-5055-4b4c-940b-35542919c646 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:08.897054 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a40d64-5055-4b4c-940b-35542919c646 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:08.898539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a40d64-5055-4b4c-940b-35542919c646 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:08.913455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-25a40d64-5055-4b4c-940b-35542919c646 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:08.913455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:08.913455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:08.913455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:08.913455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:08.913455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:08.913455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:08.913455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:08.913455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:08.913455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:08.913455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:08.913455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:08.913455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:54:08.913455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:54:08.913455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:54:08.914548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:54:08.914548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:08.914548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:08.914548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:08.914548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:08.914548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:08.914548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:08.914548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:08.914548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:08.914548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:08.914548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:08.914548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:08.914548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:08.914548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:08.914548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:08.914548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:08.914548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:08.915462 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:54:08.915462 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:54:08.915462 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:54:08.915462 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:54:08.915462 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:54:08.915462 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:54:08.915462 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:08.915462 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:08.915462 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 387/753] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:08.915462 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a40d64-5055-4b4c-940b-35542919c646 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:08.923120 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59734a25-fd22-4e4d-8960-5a892631038a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:08.923886 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59734a25-fd22-4e4d-8960-5a892631038a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:08.924178 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59734a25-fd22-4e4d-8960-5a892631038a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:08.924458 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59734a25-fd22-4e4d-8960-5a892631038a None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:08.925313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59734a25-fd22-4e4d-8960-5a892631038a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:08.925627 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59734a25-fd22-4e4d-8960-5a892631038a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:08.927201 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59734a25-fd22-4e4d-8960-5a892631038a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:08.945395 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 367/754] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Jul 11 13:54:08 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:08.946732 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7644c9-6986-42f4-8abf-81684bc75a92 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:08.947020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7644c9-6986-42f4-8abf-81684bc75a92 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:08.947300 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7644c9-6986-42f4-8abf-81684bc75a92 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:08.950437 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 368/755] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:08.950437 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7644c9-6986-42f4-8abf-81684bc75a92 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:09.024057 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:09.039374 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a40d64-5055-4b4c-940b-35542919c646 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:09.053173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:09.064317 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7644c9-6986-42f4-8abf-81684bc75a92 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:09.082754 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6aa9b-0d6d-4f9b-b893-8529585e4061 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:09.083429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6aa9b-0d6d-4f9b-b893-8529585e4061 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:09.083716 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6aa9b-0d6d-4f9b-b893-8529585e4061 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:09.083997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31f6aa9b-0d6d-4f9b-b893-8529585e4061 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:09.084727 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6aa9b-0d6d-4f9b-b893-8529585e4061 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:09.085043 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6aa9b-0d6d-4f9b-b893-8529585e4061 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:09.086424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31f6aa9b-0d6d-4f9b-b893-8529585e4061 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:09.098921 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 388/756] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:09.099554 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6aa9b-0d6d-4f9b-b893-8529585e4061 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:09.110904 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39c68298-68a5-4d1c-a79a-e78dd9a4d6db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:09.111577 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c68298-68a5-4d1c-a79a-e78dd9a4d6db None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:09.111873 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c68298-68a5-4d1c-a79a-e78dd9a4d6db None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:09.112147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39c68298-68a5-4d1c-a79a-e78dd9a4d6db None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:09.112943 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c68298-68a5-4d1c-a79a-e78dd9a4d6db None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:09.113259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c68298-68a5-4d1c-a79a-e78dd9a4d6db None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:09.114735 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39c68298-68a5-4d1c-a79a-e78dd9a4d6db None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:09.131993 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 369/757] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:54:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:09.131993 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d138f-3ff1-4e8f-b244-9a0f3076dcf3 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:09.131993 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d138f-3ff1-4e8f-b244-9a0f3076dcf3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:09.131993 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465d138f-3ff1-4e8f-b244-9a0f3076dcf3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:09.147838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-465d138f-3ff1-4e8f-b244-9a0f3076dcf3 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:09.149070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-465d138f-3ff1-4e8f-b244-9a0f3076dcf3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:09.220998 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31f6aa9b-0d6d-4f9b-b893-8529585e4061 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:09.226492 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 370/758] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:09 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:09.226492 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d138f-3ff1-4e8f-b244-9a0f3076dcf3 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:09.268170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67403962-96e4-4f87-a4be-b09fd4863819 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:09.268836 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67403962-96e4-4f87-a4be-b09fd4863819 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:09.269125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67403962-96e4-4f87-a4be-b09fd4863819 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:09.269399 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67403962-96e4-4f87-a4be-b09fd4863819 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:09.289494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:09.313194 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:09.334729 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465d138f-3ff1-4e8f-b244-9a0f3076dcf3 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:09.349923 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67403962-96e4-4f87-a4be-b09fd4863819 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:09.351490 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67403962-96e4-4f87-a4be-b09fd4863819 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:09.372069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820eee6-a021-4f2f-b2db-2b2cd326b382 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:09.373154 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820eee6-a021-4f2f-b2db-2b2cd326b382 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:09.373471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820eee6-a021-4f2f-b2db-2b2cd326b382 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:09.373751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7820eee6-a021-4f2f-b2db-2b2cd326b382 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:09.380837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820eee6-a021-4f2f-b2db-2b2cd326b382 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:09.382276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7820eee6-a021-4f2f-b2db-2b2cd326b382 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:09.396540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-7820eee6-a021-4f2f-b2db-2b2cd326b382 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:09.396540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:09.396540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:09.396540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:09.396540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:09.396540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:09.396540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:09.396540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:09.396540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:09.396540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:09.396540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:09.396540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:09.396540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:54:09.396540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:54:09.396540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:54:09.397555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:54:09.397555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:09.397555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:09.397555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:09.397555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:09.397555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:09.397555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:09.397555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:09.397555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:09.397555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:09.397555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:09.397555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:09.397555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:09.397555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:09.397555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:09.397555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:09.397555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:09.398870 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:54:09.398870 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:54:09.398870 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:54:09.398870 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:54:09.398870 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:54:09.398870 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:54:09.398870 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:09.398870 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:09.398870 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 371/759] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:09.398870 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820eee6-a021-4f2f-b2db-2b2cd326b382 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:09.465543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 389/760] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:54:09 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 13:54:09.466783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67403962-96e4-4f87-a4be-b09fd4863819 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:09.508159 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7820eee6-a021-4f2f-b2db-2b2cd326b382 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:09.518670 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:09.547143 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc2286-69b0-4f2f-a5c8-a3408cc477ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:09.548093 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc2286-69b0-4f2f-a5c8-a3408cc477ef None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:09.548406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc2286-69b0-4f2f-a5c8-a3408cc477ef None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:09.548671 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78cc2286-69b0-4f2f-a5c8-a3408cc477ef None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:09.549842 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc2286-69b0-4f2f-a5c8-a3408cc477ef None admin] RBAC: Adding query filter params (role.id=6964639a7f6c41e8a6701ef533151c41, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=d9c1369033a44c1c81d4c1c9ff2bbedb) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:09.550235 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc2286-69b0-4f2f-a5c8-a3408cc477ef None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:09.551655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78cc2286-69b0-4f2f-a5c8-a3408cc477ef None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:09.564513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 372/761] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:54:09 2022] GET /identity/v3/role_assignments?user.id=d9c1369033a44c1c81d4c1c9ff2bbedb&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 259 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:09.564513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc2286-69b0-4f2f-a5c8-a3408cc477ef None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:09.564513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:09.593763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67403962-96e4-4f87-a4be-b09fd4863819 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:09.638162 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43791531-8a06-46a0-ba51-ad39520ac46c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:09.639357 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43791531-8a06-46a0-ba51-ad39520ac46c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:09.639766 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43791531-8a06-46a0-ba51-ad39520ac46c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:09.640056 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43791531-8a06-46a0-ba51-ad39520ac46c None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:09.640816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43791531-8a06-46a0-ba51-ad39520ac46c None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:09.642697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43791531-8a06-46a0-ba51-ad39520ac46c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:09.657294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-43791531-8a06-46a0-ba51-ad39520ac46c None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 13:54:09.657294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:09.657294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:09.657294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:09.657294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:09.657294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:09.657294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:09.657294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:09.657294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 13:54:09.657294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:09.657294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:09.657294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:09.657294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:09.657294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:09.657294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:09.657294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:09.658339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:09.658339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:09.658339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:09.658339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:09.658339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:09.658339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:09.658339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:09.658339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:09.658339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:09.658339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:09.658339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:09.658339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:09.658339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:09.658339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:09.658339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:09.658339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:09.659611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:09.659611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:09.659611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:09.659611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:09.659611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:09.659611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:09.659611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:09.659611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:09.659611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:09.659611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:09.659611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:09.659611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:09.659611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:09.659611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 13:54:09.659611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:09.659611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 390/762] 10.222.0.49 () {56 vars in 1129 bytes} [Mon Jul 11 13:54:09 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:09.660802 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43791531-8a06-46a0-ba51-ad39520ac46c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:09.691865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78cc2286-69b0-4f2f-a5c8-a3408cc477ef None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:09.729656 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5addd32b-f214-4b74-be23-fe0374d16c24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:09.730396 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addd32b-f214-4b74-be23-fe0374d16c24 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:09.730690 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addd32b-f214-4b74-be23-fe0374d16c24 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:09.730977 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5addd32b-f214-4b74-be23-fe0374d16c24 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:09.731785 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5addd32b-f214-4b74-be23-fe0374d16c24 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:09.732101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5addd32b-f214-4b74-be23-fe0374d16c24 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:09.735108 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5addd32b-f214-4b74-be23-fe0374d16c24 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:09.752059 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 373/763] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:54:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:09.752908 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5addd32b-f214-4b74-be23-fe0374d16c24 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:09.785451 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43791531-8a06-46a0-ba51-ad39520ac46c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:09.799906 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:09.821428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:09.825202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20d70469-8a45-4553-a2ac-b32cf57f13c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:09.826009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d70469-8a45-4553-a2ac-b32cf57f13c3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:09.826337 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d70469-8a45-4553-a2ac-b32cf57f13c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:09.826815 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20d70469-8a45-4553-a2ac-b32cf57f13c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:09.838504 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d70469-8a45-4553-a2ac-b32cf57f13c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:09.839970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20d70469-8a45-4553-a2ac-b32cf57f13c3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:09.857795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-20d70469-8a45-4553-a2ac-b32cf57f13c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:09.857795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:09.857795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:09.857795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:09.857795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:09.857795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:09.857795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:09.857795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:09.857795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:09.857795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:09.857795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:09.857795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:09.857795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:09.857795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:09.857795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:09.860919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:09.860919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:09.860919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:09.860919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:09.860919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:09.860919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:09.860919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:09.860919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:09.860919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:09.860919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:09.860919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:09.860919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:09.860919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:09.860919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:09.860919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:09.860919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:09.860919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:09.862108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:09.862108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:09.862108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:09.862108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:09.862108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 391/764] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:09.862108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aceded-a962-4a85-870b-3905efa9e1c1 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:09.862108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aceded-a962-4a85-870b-3905efa9e1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:09.862108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44aceded-a962-4a85-870b-3905efa9e1c1 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:09.866750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 392/765] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:09 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:09.867597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aceded-a962-4a85-870b-3905efa9e1c1 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:09.869541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5addd32b-f214-4b74-be23-fe0374d16c24 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:09.909911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0008f-7c3b-4b4f-add9-75e9d33030c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:09.910623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0008f-7c3b-4b4f-add9-75e9d33030c5 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:09.910916 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0008f-7c3b-4b4f-add9-75e9d33030c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:09.911198 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45d0008f-7c3b-4b4f-add9-75e9d33030c5 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:09.912131 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0008f-7c3b-4b4f-add9-75e9d33030c5 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:09.912262 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0008f-7c3b-4b4f-add9-75e9d33030c5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:09.913611 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45d0008f-7c3b-4b4f-add9-75e9d33030c5 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:09.923531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 374/766] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Jul 11 13:54:09 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:09.923531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0008f-7c3b-4b4f-add9-75e9d33030c5 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:10.008529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44aceded-a962-4a85-870b-3905efa9e1c1 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:10.028685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45d0008f-7c3b-4b4f-add9-75e9d33030c5 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:10.056466 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90ebb60-ac7c-4ee8-8c88-9aa6919c069a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:10.057390 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ebb60-ac7c-4ee8-8c88-9aa6919c069a None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:10.057678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ebb60-ac7c-4ee8-8c88-9aa6919c069a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:10.057956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90ebb60-ac7c-4ee8-8c88-9aa6919c069a None admin] PATH_INFO: `/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/ef5dac3f311e43809a46995bab56e780/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:10.059976 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90ebb60-ac7c-4ee8-8c88-9aa6919c069a None admin] RBAC: Authorizing `identity:create_grant(project_id=08ee65f8b338491f80e7beac5e316b23, user_id=ef5dac3f311e43809a46995bab56e780, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:10.061954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90ebb60-ac7c-4ee8-8c88-9aa6919c069a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:10.065453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:10.075489 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d253d-0d94-4dc6-bcdc-7513f9128120 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:10.076095 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 393/767] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:54:09 2022] PUT /identity/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/ef5dac3f311e43809a46995bab56e780/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:54:10.076716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d253d-0d94-4dc6-bcdc-7513f9128120 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:10.077170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d253d-0d94-4dc6-bcdc-7513f9128120 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:10.077592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030e071-f098-489f-aa95-70cdddc35e21 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:10.077791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d253d-0d94-4dc6-bcdc-7513f9128120 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:10.077889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030e071-f098-489f-aa95-70cdddc35e21 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:10.078149 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c030e071-f098-489f-aa95-70cdddc35e21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:10.078994 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d253d-0d94-4dc6-bcdc-7513f9128120 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:10.079482 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d253d-0d94-4dc6-bcdc-7513f9128120 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:10.081169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d253d-0d94-4dc6-bcdc-7513f9128120 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:10.106967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c030e071-f098-489f-aa95-70cdddc35e21 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:10.109107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 375/768] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:10.109506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c030e071-f098-489f-aa95-70cdddc35e21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:10.110103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d253d-0d94-4dc6-bcdc-7513f9128120 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:10.135948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:10.208959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 394/769] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:10 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:10.210225 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030e071-f098-489f-aa95-70cdddc35e21 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:10.289971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d253d-0d94-4dc6-bcdc-7513f9128120 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:10.348668 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c030e071-f098-489f-aa95-70cdddc35e21 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:10.362650 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea4d44-e021-420d-920b-11b0d72644fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:10.363993 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea4d44-e021-420d-920b-11b0d72644fa None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:10.364501 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea4d44-e021-420d-920b-11b0d72644fa None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:10.365005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2ea4d44-e021-420d-920b-11b0d72644fa None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:10.366414 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea4d44-e021-420d-920b-11b0d72644fa None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:10.366955 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea4d44-e021-420d-920b-11b0d72644fa None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:10.369286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2ea4d44-e021-420d-920b-11b0d72644fa None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:10.379567 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 376/770] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Jul 11 13:54:10 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:10.380738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea4d44-e021-420d-920b-11b0d72644fa None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:10.394197 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d77a1-269e-469c-a444-a3e474565851 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:10.395042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d77a1-269e-469c-a444-a3e474565851 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:10.395327 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d77a1-269e-469c-a444-a3e474565851 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:10.395601 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d77a1-269e-469c-a444-a3e474565851 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:10.407927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d77a1-269e-469c-a444-a3e474565851 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:10.410609 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-135d77a1-269e-469c-a444-a3e474565851 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:10.427973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-135d77a1-269e-469c-a444-a3e474565851 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:10.427973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:10.427973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:10.427973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:10.427973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:10.427973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:10.427973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:10.427973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:10.427973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:10.427973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:10.427973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:10.427973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:10.427973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:54:10.427973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:54:10.427973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:54:10.429114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:54:10.429114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:10.429114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:10.429114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:10.429114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:10.429114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:10.429114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:10.429114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:10.429114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:10.429114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:10.429114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:10.429114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:10.429114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:10.429114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:10.429114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:10.429114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:10.429114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:10.430284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:54:10.430284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:54:10.430284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:54:10.430284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:54:10.430284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:54:10.430284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:54:10.430284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:10.430284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:10.434387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 395/771] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:10.435363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d77a1-269e-469c-a444-a3e474565851 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:10.463385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:10.470662 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:10.538522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2ea4d44-e021-420d-920b-11b0d72644fa None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:10.583969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d77a1-269e-469c-a444-a3e474565851 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:10.600458 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa00332-e6e7-4740-8a55-03733a530550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:10.601124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa00332-e6e7-4740-8a55-03733a530550 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:10.601561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa00332-e6e7-4740-8a55-03733a530550 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:10.601697 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa00332-e6e7-4740-8a55-03733a530550 None admin] PATH_INFO: `/v3/services/network` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:10.602378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa00332-e6e7-4740-8a55-03733a530550 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:10.603767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aa00332-e6e7-4740-8a55-03733a530550 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:10.616749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-6aa00332-e6e7-4740-8a55-03733a530550 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 13:54:10.616749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:10.616749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:10.616749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:10.616749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:10.616749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:10.616749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:10.616749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:10.616749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 13:54:10.616749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:10.616749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:10.616749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:10.616749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:10.616749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:10.616749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:10.616749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:10.616749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:10.617794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:10.617794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:10.617794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:10.617794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:10.617794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:10.617794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:10.617794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:10.617794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:10.617794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:10.617794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:10.617794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:10.617794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:10.617794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:10.617794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:10.617794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:10.617794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:10.618689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:10.618689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:10.618689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:10.618689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:10.618689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:10.618689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:10.618689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:10.618689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:10.618689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:10.618689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:10.618689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:10.618689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:10.618689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 13:54:10.618689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:10.618689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 377/772] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:10 2022] GET /identity/v3/services/network => generated 88 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:10.619472 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa00332-e6e7-4740-8a55-03733a530550 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:10.635222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc396e-41a7-45d3-b976-1787f9cb8653 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:10.637005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc396e-41a7-45d3-b976-1787f9cb8653 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:10.637507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc396e-41a7-45d3-b976-1787f9cb8653 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:10.638237 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bc396e-41a7-45d3-b976-1787f9cb8653 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:10.639605 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc396e-41a7-45d3-b976-1787f9cb8653 None admin] RBAC: Adding query filter params (interface=public, service_id=ecf12e27d72f4497aa5d6e4c3d5c3d5a, region_id=RegionOne) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:10.640619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc396e-41a7-45d3-b976-1787f9cb8653 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:10.642218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bc396e-41a7-45d3-b976-1787f9cb8653 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:10.655177 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 396/773] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Jul 11 13:54:10 2022] GET /identity/v3/endpoints?service_id=ecf12e27d72f4497aa5d6e4c3d5c3d5a&interface=public®ion_id=RegionOne => generated 194 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:10.656323 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc396e-41a7-45d3-b976-1787f9cb8653 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:10.737622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa00332-e6e7-4740-8a55-03733a530550 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:10.755436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:10.773955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:10.776043 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090edf76-1678-4fc2-babd-221625b6387c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:10.776942 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090edf76-1678-4fc2-babd-221625b6387c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:10.777423 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090edf76-1678-4fc2-babd-221625b6387c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:10.777960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-090edf76-1678-4fc2-babd-221625b6387c None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:10.779060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090edf76-1678-4fc2-babd-221625b6387c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:10.780111 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090edf76-1678-4fc2-babd-221625b6387c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:10.781749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-090edf76-1678-4fc2-babd-221625b6387c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:10.795677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 378/774] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:54:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:10.796958 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090edf76-1678-4fc2-babd-221625b6387c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:10.802676 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bc396e-41a7-45d3-b976-1787f9cb8653 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:10.843391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c10ed-4a13-4158-825a-2b7c6781a700 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:10.844554 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c10ed-4a13-4158-825a-2b7c6781a700 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:10.845041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c10ed-4a13-4158-825a-2b7c6781a700 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:10.845539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c28c10ed-4a13-4158-825a-2b7c6781a700 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:10.846561 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28c10ed-4a13-4158-825a-2b7c6781a700 None admin] RBAC: Adding query filter params (name=network) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:10.847086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28c10ed-4a13-4158-825a-2b7c6781a700 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:10.848928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c28c10ed-4a13-4158-825a-2b7c6781a700 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:10.855059 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 397/775] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 11 13:54:10 2022] GET /identity/v3/services?name=network => generated 124 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:10.856142 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c10ed-4a13-4158-825a-2b7c6781a700 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:10.926238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-090edf76-1678-4fc2-babd-221625b6387c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:10.963192 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175cd77-6984-4a94-b29c-477264cacf65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:10.964208 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175cd77-6984-4a94-b29c-477264cacf65 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:10.964690 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175cd77-6984-4a94-b29c-477264cacf65 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:10.965188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f175cd77-6984-4a94-b29c-477264cacf65 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:10.966086 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175cd77-6984-4a94-b29c-477264cacf65 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:10.968230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f175cd77-6984-4a94-b29c-477264cacf65 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:10.974238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 379/776] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Jul 11 13:54:10 2022] GET /identity/v3/services => generated 772 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:10.975777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175cd77-6984-4a94-b29c-477264cacf65 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:10.982717 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c28c10ed-4a13-4158-825a-2b7c6781a700 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:11.024614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e6e4e-0945-49bd-ab43-b21ed1740ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:11.025701 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e6e4e-0945-49bd-ab43-b21ed1740ca7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:11.026185 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e6e4e-0945-49bd-ab43-b21ed1740ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:11.026712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422e6e4e-0945-49bd-ab43-b21ed1740ca7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:11.027803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e6e4e-0945-49bd-ab43-b21ed1740ca7 None admin] RBAC: Adding query filter params (role.id=1f7cf26664d04e5e9331ead293476442, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=52076f59f68f4d77a0dadbf7a98d2b57) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:11.028332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e6e4e-0945-49bd-ab43-b21ed1740ca7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:11.030893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422e6e4e-0945-49bd-ab43-b21ed1740ca7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:11.031749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:11.047053 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 398/777] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:54:10 2022] GET /identity/v3/role_assignments?user.id=52076f59f68f4d77a0dadbf7a98d2b57&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=1f7cf26664d04e5e9331ead293476442 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:11.131781 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f175cd77-6984-4a94-b29c-477264cacf65 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:11.189881 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235bfffd-d54a-4022-ad08-573248204109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:11.191020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bfffd-d54a-4022-ad08-573248204109 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:11.191613 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bfffd-d54a-4022-ad08-573248204109 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:11.192103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-235bfffd-d54a-4022-ad08-573248204109 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:11.193107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235bfffd-d54a-4022-ad08-573248204109 None admin] RBAC: Adding query filter params (type=network) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:11.193639 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235bfffd-d54a-4022-ad08-573248204109 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:11.195433 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-235bfffd-d54a-4022-ad08-573248204109 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:11.202745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 380/778] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 11 13:54:10 2022] GET /identity/v3/services?type=network => generated 350 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:11.209160 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e6e4e-0945-49bd-ab43-b21ed1740ca7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:11.227475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:11.335230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422e6e4e-0945-49bd-ab43-b21ed1740ca7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:11.370181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d887bb50-cc8e-4b17-b6e7-297b4696461f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:11.370897 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887bb50-cc8e-4b17-b6e7-297b4696461f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:11.371172 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887bb50-cc8e-4b17-b6e7-297b4696461f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:11.371442 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d887bb50-cc8e-4b17-b6e7-297b4696461f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:11.372247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887bb50-cc8e-4b17-b6e7-297b4696461f None admin] RBAC: Adding query filter params (interface=public, service_id=cc2a6270cecb44b6bf6f7abfac1c91b4, region_id=RegionOne) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:11.372563 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887bb50-cc8e-4b17-b6e7-297b4696461f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:11.374499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d887bb50-cc8e-4b17-b6e7-297b4696461f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:11.380655 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 399/779] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Jul 11 13:54:11 2022] GET /identity/v3/endpoints?service_id=cc2a6270cecb44b6bf6f7abfac1c91b4&interface=public®ion_id=RegionOne => generated 194 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:11.385909 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235bfffd-d54a-4022-ad08-573248204109 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:11.631627 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-235bfffd-d54a-4022-ad08-573248204109 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:11.634190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:11.668838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d6b1e4-2904-4243-9542-4895f19ff31e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:11.669523 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6b1e4-2904-4243-9542-4895f19ff31e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:11.669809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6b1e4-2904-4243-9542-4895f19ff31e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:11.670652 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d6b1e4-2904-4243-9542-4895f19ff31e None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:11.671360 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d6b1e4-2904-4243-9542-4895f19ff31e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:11.672749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d6b1e4-2904-4243-9542-4895f19ff31e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:11.679027 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 381/780] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Jul 11 13:54:11 2022] GET /identity/v3/services => generated 772 bytes in 293 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:11.744772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50986a3-d7ba-416b-9304-4766ab1a694e None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:11.745456 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50986a3-d7ba-416b-9304-4766ab1a694e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:11.745941 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d50986a3-d7ba-416b-9304-4766ab1a694e None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:11.748029 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 400/781] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:11 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:11.756813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e131f7-9eed-46c6-b0ff-c5271d81e3a9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:11.757323 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e131f7-9eed-46c6-b0ff-c5271d81e3a9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:11.757429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e131f7-9eed-46c6-b0ff-c5271d81e3a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:11.778237 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-83e131f7-9eed-46c6-b0ff-c5271d81e3a9 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:11.779550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-83e131f7-9eed-46c6-b0ff-c5271d81e3a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:11.855804 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 382/782] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:11 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:11.895598 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50986a3-d7ba-416b-9304-4766ab1a694e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:11.987503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:12.022434 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d50986a3-d7ba-416b-9304-4766ab1a694e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:12.068243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306c303-aa85-4348-a9d7-95e6306a53a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:12.069039 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306c303-aa85-4348-a9d7-95e6306a53a1 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:12.069320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306c303-aa85-4348-a9d7-95e6306a53a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:12.069593 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306c303-aa85-4348-a9d7-95e6306a53a1 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:12.089915 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a306c303-aa85-4348-a9d7-95e6306a53a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:54:12.098453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a306c303-aa85-4348-a9d7-95e6306a53a1 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:54:12.098453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:12.098453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:12.098453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:12.098453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:12.098453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:12.098453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:12.098453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:12.098453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:12.098453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:12.098453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:12.098453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:12.098453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:54:12.098453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:54:12.098453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:54:12.098453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:54:12.099384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:54:12.099384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:54:12.099384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:54:12.099384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:54:12.099384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:12.099384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:12.099384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:12.099384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:12.099384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:12.099384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:12.099384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:12.099384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:12.099384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:12.099384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:12.099384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:12.099384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:12.100645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:12.100645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:12.100645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:12.100645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:12.100645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:54:12.100645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:54:12.100645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:54:12.100645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:54:12.100645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:54:12.100645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:54:12.100645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:54:12.100645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:12.100645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 401/783] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:54:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:12.106144 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e131f7-9eed-46c6-b0ff-c5271d81e3a9 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:12.216217 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:12.221529 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e131f7-9eed-46c6-b0ff-c5271d81e3a9 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:12.257312 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1402c-368f-4fe8-8c0c-1df713408e19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:12.257905 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1402c-368f-4fe8-8c0c-1df713408e19 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:12.258186 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1402c-368f-4fe8-8c0c-1df713408e19 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:12.258493 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc1402c-368f-4fe8-8c0c-1df713408e19 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:12.259249 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1402c-368f-4fe8-8c0c-1df713408e19 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:12.259555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1402c-368f-4fe8-8c0c-1df713408e19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:12.261090 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc1402c-368f-4fe8-8c0c-1df713408e19 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:12.270990 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 383/784] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:54:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:12.276565 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306c303-aa85-4348-a9d7-95e6306a53a1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:12.407098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a306c303-aa85-4348-a9d7-95e6306a53a1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:12.438806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:12.448158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9b48453-7632-49dd-85ba-9b73783b1646 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:12.449168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b48453-7632-49dd-85ba-9b73783b1646 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:12.449458 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b48453-7632-49dd-85ba-9b73783b1646 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:12.449736 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9b48453-7632-49dd-85ba-9b73783b1646 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:12.461656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b48453-7632-49dd-85ba-9b73783b1646 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:12.463137 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9b48453-7632-49dd-85ba-9b73783b1646 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:12.488615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c9b48453-7632-49dd-85ba-9b73783b1646 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:12.488615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:12.488615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:12.488615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:12.488615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:12.488615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:12.488615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:12.488615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:12.488615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:12.488615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:12.488615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:12.488615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:12.488615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:12.488615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:12.488615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:12.489540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:12.489540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:12.489540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:12.489540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:12.489540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:12.489540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:12.489540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:12.489540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:12.489540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:12.489540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:12.489540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:12.489540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:12.489540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:12.489540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:12.489540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:12.489540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:12.489540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:12.490725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:12.490725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:12.490725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:12.490725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:12.490725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 402/785] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:12.495928 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1402c-368f-4fe8-8c0c-1df713408e19 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:12.537507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f27b2-20c9-4b76-b24c-75a7eb58cbee None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:12.537507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f27b2-20c9-4b76-b24c-75a7eb58cbee None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:12.537507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255f27b2-20c9-4b76-b24c-75a7eb58cbee None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:12.538251 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 403/786] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:54:12.543394 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8746e12-a9a0-460d-8fea-6581e9274144 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:12.543913 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8746e12-a9a0-460d-8fea-6581e9274144 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:12.544419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8746e12-a9a0-460d-8fea-6581e9274144 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:12.572502 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d8746e12-a9a0-460d-8fea-6581e9274144 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:12.574225 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d8746e12-a9a0-460d-8fea-6581e9274144 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:12.652946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc1402c-368f-4fe8-8c0c-1df713408e19 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:12.689017 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819a0ff-b262-4b67-88bf-e786fe715f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:12.690053 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819a0ff-b262-4b67-88bf-e786fe715f46 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:12.690582 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819a0ff-b262-4b67-88bf-e786fe715f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:12.692002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3819a0ff-b262-4b67-88bf-e786fe715f46 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:12.692939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819a0ff-b262-4b67-88bf-e786fe715f46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:12.693633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819a0ff-b262-4b67-88bf-e786fe715f46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:12.696479 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3819a0ff-b262-4b67-88bf-e786fe715f46 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:12.715532 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 384/787] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:12.723338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819a0ff-b262-4b67-88bf-e786fe715f46 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:12.738747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 404/788] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:12 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:54:12.748607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:12.786261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8746e12-a9a0-460d-8fea-6581e9274144 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:12.899250 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3819a0ff-b262-4b67-88bf-e786fe715f46 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:12.976090 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cdb49-9afe-4a19-aa56-7534353467e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:12.977522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cdb49-9afe-4a19-aa56-7534353467e1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:12.978003 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cdb49-9afe-4a19-aa56-7534353467e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:12.978515 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39cdb49-9afe-4a19-aa56-7534353467e1 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:12.993172 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cdb49-9afe-4a19-aa56-7534353467e1 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:12.995576 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f39cdb49-9afe-4a19-aa56-7534353467e1 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:13.009473 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8746e12-a9a0-460d-8fea-6581e9274144 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:13.019315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-f39cdb49-9afe-4a19-aa56-7534353467e1 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 13:54:13.019315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:13.019315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:13.019315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:13.019315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:13.019315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:13.019315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:13.019315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:13.019315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:13.019315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:13.019315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:13.019315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:13.019315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:54:13.019315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:54:13.019315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:54:13.019315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:54:13.020547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:13.020547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:13.020547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:54:13.020547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:13.020547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:54:13.020547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:13.020547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:13.020547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:13.020547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:13.020547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:13.020547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:13.020547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:13.020547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:13.020547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:13.020547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:13.020547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:13.020547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:13.020547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:13.021742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:13.021742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:13.021742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:54:13.021742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:54:13.021742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:54:13.021742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:54:13.021742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:54:13.021742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:54:13.021742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 13:54:13.021742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:13.026760 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 385/789] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:54:12 2022] GET /identity/v3/users/swift => generated 83 bytes in 304 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:13.035240 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cdb49-9afe-4a19-aa56-7534353467e1 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:13.039233 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:13.086625 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e47116-ebd7-44ab-bdae-eb6481430b73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:13.087846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e47116-ebd7-44ab-bdae-eb6481430b73 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:13.088277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e47116-ebd7-44ab-bdae-eb6481430b73 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:13.088844 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e47116-ebd7-44ab-bdae-eb6481430b73 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:13.117713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e47116-ebd7-44ab-bdae-eb6481430b73 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:54:13.132505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-55e47116-ebd7-44ab-bdae-eb6481430b73 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:54:13.132505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:13.132505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:13.132505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:13.132505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:13.132505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:13.132505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:13.132505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:13.132505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:13.132505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:13.132505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:13.132505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:13.132505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:54:13.132505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:54:13.132505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:54:13.132505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:54:13.134204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:54:13.134204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:54:13.134204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:54:13.134204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:54:13.134204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:13.134204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:13.134204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:13.134204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:13.134204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:13.134204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:13.134204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:13.134204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:13.134204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:13.134204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:13.134204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:13.134204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:13.135647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:13.135647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:13.135647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:13.135647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:13.135647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:54:13.135647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:54:13.135647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:54:13.135647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:54:13.135647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:54:13.135647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:54:13.135647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:54:13.135647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:13.140493 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 405/790] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:54:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 355 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:13.148501 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e47116-ebd7-44ab-bdae-eb6481430b73 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:13.191764 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:13.214616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39cdb49-9afe-4a19-aa56-7534353467e1 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:13.254080 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada5623-3dae-4d4f-869b-a00e5c4fa919 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:13.255257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada5623-3dae-4d4f-869b-a00e5c4fa919 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:13.255763 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada5623-3dae-4d4f-869b-a00e5c4fa919 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:13.256241 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada5623-3dae-4d4f-869b-a00e5c4fa919 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:13.257364 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada5623-3dae-4d4f-869b-a00e5c4fa919 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:13.257904 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada5623-3dae-4d4f-869b-a00e5c4fa919 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:13.259629 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada5623-3dae-4d4f-869b-a00e5c4fa919 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:13.281583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 386/791] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Jul 11 13:54:13 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:13.287784 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada5623-3dae-4d4f-869b-a00e5c4fa919 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:13.316395 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e47116-ebd7-44ab-bdae-eb6481430b73 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:13.352807 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929828a9-417f-4b9c-8d93-5bbd4459da4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:13.353494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929828a9-417f-4b9c-8d93-5bbd4459da4e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:13.353776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929828a9-417f-4b9c-8d93-5bbd4459da4e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:13.354058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-929828a9-417f-4b9c-8d93-5bbd4459da4e None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:13.355295 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929828a9-417f-4b9c-8d93-5bbd4459da4e None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:13.355295 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929828a9-417f-4b9c-8d93-5bbd4459da4e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:13.356547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-929828a9-417f-4b9c-8d93-5bbd4459da4e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:13.388603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 406/792] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:54:13 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:13.391374 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929828a9-417f-4b9c-8d93-5bbd4459da4e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:13.416767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada5623-3dae-4d4f-869b-a00e5c4fa919 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:13.454827 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:13.458389 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf8c75-2878-4c92-b872-78a6534465ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:13.459617 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf8c75-2878-4c92-b872-78a6534465ab None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:13.460099 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf8c75-2878-4c92-b872-78a6534465ab None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:13.460632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cf8c75-2878-4c92-b872-78a6534465ab None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:13.470996 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:13.477363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf8c75-2878-4c92-b872-78a6534465ab None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:13.479773 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cf8c75-2878-4c92-b872-78a6534465ab None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:13.500802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-69cf8c75-2878-4c92-b872-78a6534465ab None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:13.500802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:13.500802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:13.500802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:13.500802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:13.500802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:13.500802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:13.500802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:13.500802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:13.500802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:13.500802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:13.500802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:13.500802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:13.500802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:13.500802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:13.501766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:13.501766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:13.501766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:13.501766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:13.501766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:13.501766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:13.501766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:13.501766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:13.501766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:13.501766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:13.501766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:13.501766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:13.501766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:13.501766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:13.501766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:13.501766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:13.501766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:13.502653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:13.502653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:13.502653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:13.502653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:13.505090 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 387/793] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:13.511553 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf8c75-2878-4c92-b872-78a6534465ab None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:13.630716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cf8c75-2878-4c92-b872-78a6534465ab None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:13.673340 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d022ca-c5c5-48ab-a993-e616a309e38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:13.674027 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d022ca-c5c5-48ab-a993-e616a309e38f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:13.674339 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d022ca-c5c5-48ab-a993-e616a309e38f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:13.674605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d022ca-c5c5-48ab-a993-e616a309e38f None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:13.675341 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d022ca-c5c5-48ab-a993-e616a309e38f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:13.675649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d022ca-c5c5-48ab-a993-e616a309e38f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:13.677037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d022ca-c5c5-48ab-a993-e616a309e38f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:13.686903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-929828a9-417f-4b9c-8d93-5bbd4459da4e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:13.693241 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 388/794] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:13.699441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c7c0b-8c5c-4bf5-88a4-cec5e17430b2 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:13.701698 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c7c0b-8c5c-4bf5-88a4-cec5e17430b2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:13.701984 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7c7c0b-8c5c-4bf5-88a4-cec5e17430b2 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:13.703780 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 389/795] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:13 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:13.704566 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c7c0b-8c5c-4bf5-88a4-cec5e17430b2 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:13.737893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c64d2a20-4041-423a-879f-c38c9907f9c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:13.738932 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d2a20-4041-423a-879f-c38c9907f9c6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:13.739555 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d2a20-4041-423a-879f-c38c9907f9c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:13.739906 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c64d2a20-4041-423a-879f-c38c9907f9c6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:13.758016 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d2a20-4041-423a-879f-c38c9907f9c6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:13.760513 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c64d2a20-4041-423a-879f-c38c9907f9c6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:13.769705 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:13.790157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c64d2a20-4041-423a-879f-c38c9907f9c6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:13.790157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:13.790157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:13.790157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:13.790157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:13.790157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:13.790157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:13.790157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:13.790157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:13.790157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:13.790157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:13.790157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:13.790157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:13.790157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:13.790157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:13.791373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:13.791373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:13.791373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:13.791373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:13.791373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:13.791373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:13.791373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:13.791373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:13.791373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:13.791373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:13.791373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:13.791373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:13.791373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:13.791373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:13.791373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:13.791373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:13.791373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:13.792397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:13.792397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:13.792397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:13.792397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:13.800335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 407/796] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 405 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:13.802423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92096ca7-4eb0-479c-90bf-a4f966bc185b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:13.802881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92096ca7-4eb0-479c-90bf-a4f966bc185b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:13.804338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92096ca7-4eb0-479c-90bf-a4f966bc185b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:13.828605 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-92096ca7-4eb0-479c-90bf-a4f966bc185b None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:13.830150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-92096ca7-4eb0-479c-90bf-a4f966bc185b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:13.840559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7c7c0b-8c5c-4bf5-88a4-cec5e17430b2 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:13.885843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b69dd992-0573-4c88-b5cd-0ce34c84465e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:13.887191 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dd992-0573-4c88-b5cd-0ce34c84465e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:13.887678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dd992-0573-4c88-b5cd-0ce34c84465e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:13.888161 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b69dd992-0573-4c88-b5cd-0ce34c84465e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:13.903147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dd992-0573-4c88-b5cd-0ce34c84465e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:13.905984 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b69dd992-0573-4c88-b5cd-0ce34c84465e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:13.908640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 408/797] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:13 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:13.909504 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92096ca7-4eb0-479c-90bf-a4f966bc185b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:13.926430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-b69dd992-0573-4c88-b5cd-0ce34c84465e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:13.926430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:13.926430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:13.926430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:13.926430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:13.926430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:13.926430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:13.926430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:13.926430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:13.926430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:13.926430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:13.926430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:13.926430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:54:13.926430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:54:13.926430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:54:13.927702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:54:13.927702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:13.927702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:13.927702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:13.927702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:13.927702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:13.927702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:13.927702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:13.927702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:13.927702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:13.927702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:13.927702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:13.927702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:13.927702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:13.927702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:13.927702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:13.927702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:13.928579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:54:13.928579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:54:13.928579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:54:13.928579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:54:13.928579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:54:13.928579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:54:13.928579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:13.928579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:13.928579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 390/798] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:13.928579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb3d2d-e32a-46f4-8674-d5305522784f None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:13.928579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb3d2d-e32a-46f4-8674-d5305522784f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:13.929166 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05fb3d2d-e32a-46f4-8674-d5305522784f None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:13.929166 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 391/799] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:13.933914 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb3d2d-e32a-46f4-8674-d5305522784f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:13.934919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:14.042930 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92096ca7-4eb0-479c-90bf-a4f966bc185b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:14.061519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05fb3d2d-e32a-46f4-8674-d5305522784f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:14.074196 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:14.090923 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2373f0-8d48-424d-bff0-3af80881e89f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:14.091715 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2373f0-8d48-424d-bff0-3af80881e89f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:14.092117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2373f0-8d48-424d-bff0-3af80881e89f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:14.092474 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2373f0-8d48-424d-bff0-3af80881e89f None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:14.093319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2373f0-8d48-424d-bff0-3af80881e89f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:14.093637 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2373f0-8d48-424d-bff0-3af80881e89f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:14.095059 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2373f0-8d48-424d-bff0-3af80881e89f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:14.106035 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0e15548-04dd-4104-9404-e02abfcea93b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:14.106869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e15548-04dd-4104-9404-e02abfcea93b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:14.107225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e15548-04dd-4104-9404-e02abfcea93b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:14.108266 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0e15548-04dd-4104-9404-e02abfcea93b None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:14.109021 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e15548-04dd-4104-9404-e02abfcea93b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:14.109365 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e15548-04dd-4104-9404-e02abfcea93b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:14.110912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 409/800] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:14.111668 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2373f0-8d48-424d-bff0-3af80881e89f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:14.115348 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0e15548-04dd-4104-9404-e02abfcea93b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:14.134924 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 392/801] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:54:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:14.137977 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57046559-656c-4804-8f41-38c7e3f5d604 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:14.138613 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57046559-656c-4804-8f41-38c7e3f5d604 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:14.139602 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57046559-656c-4804-8f41-38c7e3f5d604 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:14.166253 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-57046559-656c-4804-8f41-38c7e3f5d604 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:14.167724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-57046559-656c-4804-8f41-38c7e3f5d604 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:14.229386 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:14.257087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2373f0-8d48-424d-bff0-3af80881e89f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:14.267693 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 393/802] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:14.275670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57046559-656c-4804-8f41-38c7e3f5d604 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:14.299770 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d339c-35b1-4d72-9ff6-07eee9c5966d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:14.300645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d339c-35b1-4d72-9ff6-07eee9c5966d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:14.301248 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d339c-35b1-4d72-9ff6-07eee9c5966d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:14.301387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7d339c-35b1-4d72-9ff6-07eee9c5966d None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:14.313227 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7d339c-35b1-4d72-9ff6-07eee9c5966d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:54:14.321409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-6e7d339c-35b1-4d72-9ff6-07eee9c5966d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:54:14.321409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:14.321409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:14.321409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:14.321409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:14.321409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:14.321409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:14.321409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:14.321409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:14.321409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:14.321409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:14.321409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:14.321409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:54:14.321409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:54:14.321409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:54:14.321409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:54:14.322539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:54:14.322539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:54:14.322539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:54:14.322539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:54:14.322539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:14.322539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:14.322539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:14.322539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:14.322539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:14.322539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:14.322539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:14.322539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:14.322539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:14.322539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:14.322539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:14.322539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:14.323750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:14.323750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:14.323750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:14.323750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:14.323750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:54:14.323750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:54:14.323750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:54:14.323750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:54:14.323750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:54:14.323750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:54:14.323750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:54:14.323750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:14.323750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 410/803] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:54:14 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:14.323750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d339c-35b1-4d72-9ff6-07eee9c5966d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:14.403094 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:14.406691 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57046559-656c-4804-8f41-38c7e3f5d604 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:14.434270 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7d339c-35b1-4d72-9ff6-07eee9c5966d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:14.441060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc518d7-0fb1-47ce-976a-045dfb74ac1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:14.441857 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc518d7-0fb1-47ce-976a-045dfb74ac1a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:14.442143 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc518d7-0fb1-47ce-976a-045dfb74ac1a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:14.442455 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc518d7-0fb1-47ce-976a-045dfb74ac1a None admin] PATH_INFO: `/v3/users/nova` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:14.449546 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc518d7-0fb1-47ce-976a-045dfb74ac1a None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:14.451018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc518d7-0fb1-47ce-976a-045dfb74ac1a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:14.465254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-8dc518d7-0fb1-47ce-976a-045dfb74ac1a None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 13:54:14.465254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:14.465254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:14.465254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:14.465254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:14.465254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:14.465254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:14.465254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:14.465254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:14.465254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:14.465254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:14.465254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:14.465254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:54:14.465254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:54:14.465254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:54:14.465254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:54:14.466419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:14.466419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:14.466419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:54:14.466419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:14.466419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:54:14.466419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:14.466419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:14.466419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:14.466419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:14.466419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:14.466419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:14.466419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:14.466419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:14.466419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:14.466419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:14.466419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:14.466419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:14.466419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:14.467899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:14.467899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:14.467899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:54:14.467899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:54:14.467899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:54:14.467899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:54:14.467899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:54:14.467899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:54:14.467899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 13:54:14.467899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:14.467899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 394/804] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Jul 11 13:54:14 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:14.467899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc518d7-0fb1-47ce-976a-045dfb74ac1a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:14.473120 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a9d95f-3dc4-44e2-81d3-8eba29bfa64b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:14.474015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9d95f-3dc4-44e2-81d3-8eba29bfa64b None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:14.474295 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9d95f-3dc4-44e2-81d3-8eba29bfa64b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:14.474591 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67a9d95f-3dc4-44e2-81d3-8eba29bfa64b None admin] PATH_INFO: `/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/d9c1369033a44c1c81d4c1c9ff2bbedb/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:14.476605 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a9d95f-3dc4-44e2-81d3-8eba29bfa64b None admin] RBAC: Authorizing `identity:create_grant(project_id=08ee65f8b338491f80e7beac5e316b23, user_id=d9c1369033a44c1c81d4c1c9ff2bbedb, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:14.478036 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67a9d95f-3dc4-44e2-81d3-8eba29bfa64b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:14.481236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:14.493372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 411/805] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:54:14 2022] PUT /identity/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/d9c1369033a44c1c81d4c1c9ff2bbedb/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:54:14.494232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a9d95f-3dc4-44e2-81d3-8eba29bfa64b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:14.586023 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc518d7-0fb1-47ce-976a-045dfb74ac1a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:14.613353 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67a9d95f-3dc4-44e2-81d3-8eba29bfa64b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:14.630116 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56efad88-9fb0-4f2e-8371-76d6b71d2389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:14.630995 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efad88-9fb0-4f2e-8371-76d6b71d2389 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:14.631280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efad88-9fb0-4f2e-8371-76d6b71d2389 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:14.631824 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56efad88-9fb0-4f2e-8371-76d6b71d2389 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:14.632571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56efad88-9fb0-4f2e-8371-76d6b71d2389 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:14.634361 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56efad88-9fb0-4f2e-8371-76d6b71d2389 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:14.650616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-56efad88-9fb0-4f2e-8371-76d6b71d2389 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 13:54:14.650616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:14.650616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:14.650616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:14.650616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:14.650616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:14.650616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:14.650616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:14.650616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 13:54:14.650616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:14.650616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:14.650616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:14.650616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:14.650616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:14.650616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:14.650616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:14.651601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:14.651601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:14.651601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:14.651601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:14.651601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:14.651601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:14.651601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:14.651601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:14.651601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:14.651601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:14.651601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:14.651601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:14.651601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:14.651601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:14.651601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:14.651601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:14.652651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:14.652651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:14.652651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:14.652651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:14.652651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:14.652651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:14.652651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:14.652651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:14.652651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:14.652651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:14.652651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:14.652651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:14.652651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:14.652651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 11 13:54:14.652651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:14.655810 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 395/806] 10.222.0.49 () {56 vars in 1129 bytes} [Mon Jul 11 13:54:14 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:14.657467 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de622243-fcda-4d70-8ec6-08652e72e3c7 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:14.657752 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de622243-fcda-4d70-8ec6-08652e72e3c7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:14.658028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de622243-fcda-4d70-8ec6-08652e72e3c7 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:14.658156 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197301f1-fe4a-4ada-80e3-b99da54f055a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:14.658967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197301f1-fe4a-4ada-80e3-b99da54f055a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:14.659460 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 396/807] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:14.659568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197301f1-fe4a-4ada-80e3-b99da54f055a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:14.659785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-197301f1-fe4a-4ada-80e3-b99da54f055a None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:14.660044 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de622243-fcda-4d70-8ec6-08652e72e3c7 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:14.660734 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197301f1-fe4a-4ada-80e3-b99da54f055a None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:14.661141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197301f1-fe4a-4ada-80e3-b99da54f055a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:14.662637 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-197301f1-fe4a-4ada-80e3-b99da54f055a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:14.672357 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 412/808] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:54:14 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:14.673319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197301f1-fe4a-4ada-80e3-b99da54f055a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:14.692520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:14.737529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:14.778715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de622243-fcda-4d70-8ec6-08652e72e3c7 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:14.782965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-197301f1-fe4a-4ada-80e3-b99da54f055a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:14.817414 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f00cc9-02a5-4976-a6fa-76f93e1c2363 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:14.818028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7677836c-bbaa-4951-a491-f966798101ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:14.818175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f00cc9-02a5-4976-a6fa-76f93e1c2363 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:14.818363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f00cc9-02a5-4976-a6fa-76f93e1c2363 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:14.818634 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f00cc9-02a5-4976-a6fa-76f93e1c2363 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:14.818755 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677836c-bbaa-4951-a491-f966798101ce None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:14.818985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677836c-bbaa-4951-a491-f966798101ce None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:14.819257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7677836c-bbaa-4951-a491-f966798101ce None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:14.819481 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f00cc9-02a5-4976-a6fa-76f93e1c2363 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:14.819793 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f00cc9-02a5-4976-a6fa-76f93e1c2363 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:14.819964 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677836c-bbaa-4951-a491-f966798101ce None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:14.820270 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677836c-bbaa-4951-a491-f966798101ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:14.821212 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f00cc9-02a5-4976-a6fa-76f93e1c2363 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:14.821604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7677836c-bbaa-4951-a491-f966798101ce None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:14.827259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 413/809] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Jul 11 13:54:14 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:14.832665 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41924d-2ce3-4a13-8635-9cbfdd0b9bfc None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:14.833202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41924d-2ce3-4a13-8635-9cbfdd0b9bfc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:14.833792 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a41924d-2ce3-4a13-8635-9cbfdd0b9bfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:14.840127 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 397/810] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Jul 11 13:54:14 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:14.840907 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f00cc9-02a5-4976-a6fa-76f93e1c2363 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:14.856455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8a41924d-2ce3-4a13-8635-9cbfdd0b9bfc None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:14.858643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8a41924d-2ce3-4a13-8635-9cbfdd0b9bfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:14.928118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:14.935500 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 414/811] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:14 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:14.937056 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41924d-2ce3-4a13-8635-9cbfdd0b9bfc None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:14.968064 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f00cc9-02a5-4976-a6fa-76f93e1c2363 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:14.999438 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:15.009184 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab07d8-8ffb-460d-87b3-7d871607497e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:15.009986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab07d8-8ffb-460d-87b3-7d871607497e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:15.010265 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab07d8-8ffb-460d-87b3-7d871607497e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:15.010563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab07d8-8ffb-460d-87b3-7d871607497e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:15.022547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab07d8-8ffb-460d-87b3-7d871607497e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:15.023998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab07d8-8ffb-460d-87b3-7d871607497e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:15.043164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-ccab07d8-8ffb-460d-87b3-7d871607497e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:15.043164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:15.043164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:15.043164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:15.043164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:15.043164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:15.043164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:15.043164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:15.043164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:15.043164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:15.043164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:15.043164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:15.043164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:15.043164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:15.043164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:15.044105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:15.044105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:15.044105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:15.044105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:15.044105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:15.044105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:15.044105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:15.044105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:15.044105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:15.044105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:15.044105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:15.044105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:15.044105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:15.044105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:15.044105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:15.044105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:15.044105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:15.045045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:15.045045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:15.045045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:15.045045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:15.045045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 398/812] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:14 2022] GET /identity/v3/domains/Default => generated 87 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:15.045335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab07d8-8ffb-460d-87b3-7d871607497e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:15.061539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a41924d-2ce3-4a13-8635-9cbfdd0b9bfc None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:15.104999 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6386b-5c74-4919-8ea7-f580197099fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:15.105596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6386b-5c74-4919-8ea7-f580197099fb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:15.105884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6386b-5c74-4919-8ea7-f580197099fb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:15.106164 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb6386b-5c74-4919-8ea7-f580197099fb None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:15.106937 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6386b-5c74-4919-8ea7-f580197099fb None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:15.107248 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6386b-5c74-4919-8ea7-f580197099fb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:15.108622 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb6386b-5c74-4919-8ea7-f580197099fb None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:15.114749 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 415/813] 10.222.0.49 () {56 vars in 1124 bytes} [Mon Jul 11 13:54:14 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:15.115548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6386b-5c74-4919-8ea7-f580197099fb None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:15.169585 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab07d8-8ffb-460d-87b3-7d871607497e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:15.208246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d2b8c-d92f-4379-96d2-58c5b5ac8428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:15.208900 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d2b8c-d92f-4379-96d2-58c5b5ac8428 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:15.209182 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d2b8c-d92f-4379-96d2-58c5b5ac8428 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:15.209465 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692d2b8c-d92f-4379-96d2-58c5b5ac8428 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:15.212648 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:15.231338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb6386b-5c74-4919-8ea7-f580197099fb None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:15.252986 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:15.273681 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092cbb3-4cd8-48dd-b390-342a86770f2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:15.274634 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092cbb3-4cd8-48dd-b390-342a86770f2c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:15.274993 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092cbb3-4cd8-48dd-b390-342a86770f2c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:15.275344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7092cbb3-4cd8-48dd-b390-342a86770f2c None admin] PATH_INFO: `/v3/services/network` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:15.276091 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092cbb3-4cd8-48dd-b390-342a86770f2c None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:15.277470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7092cbb3-4cd8-48dd-b390-342a86770f2c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:15.288845 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d2b8c-d92f-4379-96d2-58c5b5ac8428 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:15.289455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-7092cbb3-4cd8-48dd-b390-342a86770f2c None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 13:54:15.289455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:15.289455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:15.289455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:15.289455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:15.289455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:15.289455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:15.289455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:15.289455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 13:54:15.289455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:15.289455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:15.289455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:15.289455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:15.289455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:15.289455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:15.289455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:15.289455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:15.290671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:15.290671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:15.290671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:15.290671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:15.290671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:15.290671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:15.290671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:15.290671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:15.290671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:15.290671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:15.290671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:15.290671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:15.290671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:15.290671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:15.290671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:15.290671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:15.291723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:15.291723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:15.291723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:15.291723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:15.291723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:15.291723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:15.291723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:15.291723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:15.291723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:15.291723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:15.291723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:15.291723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:15.291723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 11 13:54:15.291723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:15.292644 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692d2b8c-d92f-4379-96d2-58c5b5ac8428 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:15.296125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 416/814] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:15 2022] GET /identity/v3/services/network => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:15.296866 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092cbb3-4cd8-48dd-b390-342a86770f2c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:15.404806 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 399/815] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:54:15 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 360 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 13:54:15.405644 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d2b8c-d92f-4379-96d2-58c5b5ac8428 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:15.412378 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7092cbb3-4cd8-48dd-b390-342a86770f2c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:15.454569 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:15.473444 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ab26d-1bf8-4bac-8d4e-a3ded2cc0b44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:15.474130 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ab26d-1bf8-4bac-8d4e-a3ded2cc0b44 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:15.474439 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ab26d-1bf8-4bac-8d4e-a3ded2cc0b44 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:15.474716 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35ab26d-1bf8-4bac-8d4e-a3ded2cc0b44 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:15.475463 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ab26d-1bf8-4bac-8d4e-a3ded2cc0b44 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:15.475767 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ab26d-1bf8-4bac-8d4e-a3ded2cc0b44 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:15.477161 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35ab26d-1bf8-4bac-8d4e-a3ded2cc0b44 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:15.490428 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 417/816] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:15.491259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ab26d-1bf8-4bac-8d4e-a3ded2cc0b44 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:15.545943 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:15.550942 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692d2b8c-d92f-4379-96d2-58c5b5ac8428 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:15.592408 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583459ba-5141-4ce3-a7f3-f20a4d9bb055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:15.593140 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583459ba-5141-4ce3-a7f3-f20a4d9bb055 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:15.593425 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583459ba-5141-4ce3-a7f3-f20a4d9bb055 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:15.593745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583459ba-5141-4ce3-a7f3-f20a4d9bb055 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:15.594528 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583459ba-5141-4ce3-a7f3-f20a4d9bb055 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:15.595920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583459ba-5141-4ce3-a7f3-f20a4d9bb055 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:15.625162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 400/817] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Jul 11 13:54:15 2022] POST /identity/v3/endpoints => generated 346 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:54:15.626083 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583459ba-5141-4ce3-a7f3-f20a4d9bb055 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:15.639268 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35ab26d-1bf8-4bac-8d4e-a3ded2cc0b44 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:15.689971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9590df4-4b91-44e1-ab2b-97c69963b388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:15.690854 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9590df4-4b91-44e1-ab2b-97c69963b388 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:15.691209 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9590df4-4b91-44e1-ab2b-97c69963b388 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:15.691584 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9590df4-4b91-44e1-ab2b-97c69963b388 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:15.692469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9590df4-4b91-44e1-ab2b-97c69963b388 None admin] RBAC: Adding query filter params (name=network) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:15.692847 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9590df4-4b91-44e1-ab2b-97c69963b388 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:15.694544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9590df4-4b91-44e1-ab2b-97c69963b388 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:15.700685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 418/818] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 11 13:54:15 2022] GET /identity/v3/services?name=network => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:15.701779 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9590df4-4b91-44e1-ab2b-97c69963b388 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:15.780702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:15.798029 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583459ba-5141-4ce3-a7f3-f20a4d9bb055 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:15.840075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3cd052-1604-4cad-9296-fb7219514a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:15.840935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cd052-1604-4cad-9296-fb7219514a11 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:15.841223 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cd052-1604-4cad-9296-fb7219514a11 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:15.841495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd3cd052-1604-4cad-9296-fb7219514a11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:15.856960 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:15.860185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3cd052-1604-4cad-9296-fb7219514a11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:15.860643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9590df4-4b91-44e1-ab2b-97c69963b388 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:15.862047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd3cd052-1604-4cad-9296-fb7219514a11 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:15.882057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-fd3cd052-1604-4cad-9296-fb7219514a11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:15.882057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:15.882057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:15.882057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:15.882057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:15.882057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:15.882057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:15.882057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:15.882057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:15.882057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:15.882057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:15.882057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:15.882057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:15.882057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:15.882057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:15.883092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:15.883092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:15.883092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:15.883092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:15.883092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:15.883092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:15.883092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:15.883092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:15.883092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:15.883092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:15.883092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:15.883092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:15.883092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:15.883092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:15.883092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:15.883092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:15.883092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:15.883978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:15.883978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:15.883978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:15.883978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:15.883978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 401/819] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:15.884469 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3cd052-1604-4cad-9296-fb7219514a11 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:15.933942 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca0204-e377-4e47-be37-9b20b12ebea4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:15.950672 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca0204-e377-4e47-be37-9b20b12ebea4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:15.950672 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca0204-e377-4e47-be37-9b20b12ebea4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:15.950672 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ca0204-e377-4e47-be37-9b20b12ebea4 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:15.954333 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca0204-e377-4e47-be37-9b20b12ebea4 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:15.954333 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ca0204-e377-4e47-be37-9b20b12ebea4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:15.976592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b5ca0204-e377-4e47-be37-9b20b12ebea4 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 13:54:15.976592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:15.976592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:15.976592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:15.976592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:15.976592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:15.976592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:15.976592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:15.976592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:15.976592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:15.976592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:15.976592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:15.976592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:54:15.976592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:54:15.976592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:54:15.976592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:54:15.978507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:15.978507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:15.978507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:54:15.978507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:15.978507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:54:15.978507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:15.978507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:15.978507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:15.978507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:15.978507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:15.978507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:15.978507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:15.978507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:15.978507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:15.978507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:15.978507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:15.978507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:15.978507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:15.979947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:15.979947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:15.979947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:54:15.979947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:54:15.979947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:54:15.979947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:54:15.979947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:54:15.979947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:54:15.979947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 13:54:15.979947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:15.979947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 419/820] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 11 13:54:15 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 274 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:15.979947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca0204-e377-4e47-be37-9b20b12ebea4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:16.033553 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd3cd052-1604-4cad-9296-fb7219514a11 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:16.080094 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe4a62-25dd-49e1-9da7-63b70dae23e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:16.080786 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe4a62-25dd-49e1-9da7-63b70dae23e0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:16.081073 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe4a62-25dd-49e1-9da7-63b70dae23e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:16.081350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe4a62-25dd-49e1-9da7-63b70dae23e0 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:16.082089 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe4a62-25dd-49e1-9da7-63b70dae23e0 None admin] RBAC: Adding query filter params (type=network) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:16.082779 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe4a62-25dd-49e1-9da7-63b70dae23e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:16.084206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe4a62-25dd-49e1-9da7-63b70dae23e0 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:16.097226 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 402/821] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 11 13:54:15 2022] GET /identity/v3/services?type=network => generated 350 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:16.098424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe4a62-25dd-49e1-9da7-63b70dae23e0 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:16.111537 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:16.172740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ca0204-e377-4e47-be37-9b20b12ebea4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:16.220752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a86dc8-cc20-4677-b5e6-ffc0bab9ce6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:16.221512 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a86dc8-cc20-4677-b5e6-ffc0bab9ce6b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:16.221829 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a86dc8-cc20-4677-b5e6-ffc0bab9ce6b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:16.222228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18a86dc8-cc20-4677-b5e6-ffc0bab9ce6b None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:16.222990 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a86dc8-cc20-4677-b5e6-ffc0bab9ce6b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:16.223300 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a86dc8-cc20-4677-b5e6-ffc0bab9ce6b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:16.225398 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18a86dc8-cc20-4677-b5e6-ffc0bab9ce6b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:16.233978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe4a62-25dd-49e1-9da7-63b70dae23e0 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:16.245976 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 420/822] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:16.245976 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a86dc8-cc20-4677-b5e6-ffc0bab9ce6b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:16.274272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:16.280187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005fb83-7f19-4b8d-a0ed-c2903e842dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:16.281185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005fb83-7f19-4b8d-a0ed-c2903e842dbd None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:16.281674 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005fb83-7f19-4b8d-a0ed-c2903e842dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:16.282165 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e005fb83-7f19-4b8d-a0ed-c2903e842dbd None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:16.283248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e005fb83-7f19-4b8d-a0ed-c2903e842dbd None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:16.283768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e005fb83-7f19-4b8d-a0ed-c2903e842dbd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:16.285422 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e005fb83-7f19-4b8d-a0ed-c2903e842dbd None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:16.305520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 403/823] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Jul 11 13:54:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:16.306754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005fb83-7f19-4b8d-a0ed-c2903e842dbd None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:16.397947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18a86dc8-cc20-4677-b5e6-ffc0bab9ce6b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:16.402437 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:16.459596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e005fb83-7f19-4b8d-a0ed-c2903e842dbd None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:16.463196 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d42d44-0920-4c35-b9f5-5d5d320132e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:16.464408 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d42d44-0920-4c35-b9f5-5d5d320132e5 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:16.464973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d42d44-0920-4c35-b9f5-5d5d320132e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:16.465478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d42d44-0920-4c35-b9f5-5d5d320132e5 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:16.467014 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d42d44-0920-4c35-b9f5-5d5d320132e5 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:16.469765 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d42d44-0920-4c35-b9f5-5d5d320132e5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:16.505015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f3390-e5f1-4d5f-8311-154606c3727c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:16.506199 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f3390-e5f1-4d5f-8311-154606c3727c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:16.506754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f3390-e5f1-4d5f-8311-154606c3727c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:16.507326 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1f3390-e5f1-4d5f-8311-154606c3727c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:16.514398 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 421/824] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Jul 11 13:54:16 2022] POST /identity/v3/endpoints => generated 327 bytes in 267 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:54:16.514398 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d42d44-0920-4c35-b9f5-5d5d320132e5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:16.522383 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f3390-e5f1-4d5f-8311-154606c3727c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:16.530145 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1f3390-e5f1-4d5f-8311-154606c3727c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:16.551832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-0c1f3390-e5f1-4d5f-8311-154606c3727c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:16.551832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:16.551832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:16.551832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:16.551832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:16.551832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:16.551832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:16.551832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:16.551832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:16.551832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:16.551832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:16.551832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:16.551832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:54:16.551832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:54:16.551832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:54:16.552816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:54:16.552816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:16.552816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:16.552816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:16.552816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:16.552816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:16.552816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:16.552816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:16.552816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:16.552816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:16.552816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:16.552816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:16.552816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:16.552816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:16.552816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:16.552816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:16.552816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:16.553678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:54:16.553678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:54:16.553678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:54:16.553678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:54:16.553678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:54:16.553678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:54:16.553678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:16.553678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:16.557213 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 404/825] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:16 2022] GET /identity/v3/projects/service => generated 88 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:16.570327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f3390-e5f1-4d5f-8311-154606c3727c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:16.722763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:16.731457 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d42d44-0920-4c35-b9f5-5d5d320132e5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:16.757188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1f3390-e5f1-4d5f-8311-154606c3727c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:16.785099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afe00ec9-36b1-4f4f-9c21-4424b3a0fc2c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:16.786095 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe00ec9-36b1-4f4f-9c21-4424b3a0fc2c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:16.786419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe00ec9-36b1-4f4f-9c21-4424b3a0fc2c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:16.787017 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afe00ec9-36b1-4f4f-9c21-4424b3a0fc2c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:16.799756 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:16.802027 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe00ec9-36b1-4f4f-9c21-4424b3a0fc2c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:16.803863 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afe00ec9-36b1-4f4f-9c21-4424b3a0fc2c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:16.816327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57545ceb-c8ae-489c-b9d9-25bee6ab3cb3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:16.817397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57545ceb-c8ae-489c-b9d9-25bee6ab3cb3 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:16.817904 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57545ceb-c8ae-489c-b9d9-25bee6ab3cb3 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:16.818191 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57545ceb-c8ae-489c-b9d9-25bee6ab3cb3 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:16.819062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57545ceb-c8ae-489c-b9d9-25bee6ab3cb3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:16.819570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57545ceb-c8ae-489c-b9d9-25bee6ab3cb3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:16.820825 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57545ceb-c8ae-489c-b9d9-25bee6ab3cb3 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:16.827436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-afe00ec9-36b1-4f4f-9c21-4424b3a0fc2c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:16.827436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:16.827436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:16.827436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:16.827436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:16.827436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:16.827436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:16.827436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:16.827436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:16.827436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:16.827436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:16.827436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:16.827436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:16.827436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:16.827436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:16.828461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:16.828461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:16.828461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:16.828461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:16.828461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:16.828461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:16.828461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:16.828461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:16.828461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:16.828461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:16.828461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:16.828461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:16.828461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:16.828461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:16.828461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:16.828461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:16.828461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:16.829495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:16.829495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:16.829495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:16.829495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:16.829495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 422/826] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 319 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:16.834385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d56c51-c9ac-4b9e-a05b-c0663303ff62 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:16.835201 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d56c51-c9ac-4b9e-a05b-c0663303ff62 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:16.835381 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 405/827] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:54:16 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:16.835874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d56c51-c9ac-4b9e-a05b-c0663303ff62 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:16.836252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57545ceb-c8ae-489c-b9d9-25bee6ab3cb3 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:16.837796 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 423/828] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:16 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:16.841035 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d56c51-c9ac-4b9e-a05b-c0663303ff62 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:16.968291 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d56c51-c9ac-4b9e-a05b-c0663303ff62 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:16.970715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57545ceb-c8ae-489c-b9d9-25bee6ab3cb3 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:17.006352 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:17.019133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5effd0e4-11de-42fb-a56f-c56ec75d893b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:17.020153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effd0e4-11de-42fb-a56f-c56ec75d893b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:17.020653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effd0e4-11de-42fb-a56f-c56ec75d893b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:17.020936 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5effd0e4-11de-42fb-a56f-c56ec75d893b None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:17.021687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effd0e4-11de-42fb-a56f-c56ec75d893b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:17.022002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effd0e4-11de-42fb-a56f-c56ec75d893b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:17.023432 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5effd0e4-11de-42fb-a56f-c56ec75d893b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:17.031848 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dff40c-c158-4312-8b0b-a34b4e8b8fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:17.032525 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dff40c-c158-4312-8b0b-a34b4e8b8fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:17.032859 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dff40c-c158-4312-8b0b-a34b4e8b8fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:17.033147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85dff40c-c158-4312-8b0b-a34b4e8b8fb4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:17.034002 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dff40c-c158-4312-8b0b-a34b4e8b8fb4 None admin] RBAC: Adding query filter params (role.id=6964639a7f6c41e8a6701ef533151c41, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=ef5dac3f311e43809a46995bab56e780) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:17.034358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dff40c-c158-4312-8b0b-a34b4e8b8fb4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:17.035803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85dff40c-c158-4312-8b0b-a34b4e8b8fb4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:17.041824 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 406/829] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:17.041824 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348809c-c3ed-4c67-a71a-728c956d177a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:17.041824 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348809c-c3ed-4c67-a71a-728c956d177a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:17.041824 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2348809c-c3ed-4c67-a71a-728c956d177a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:17.049418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 424/830] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:54:16 2022] GET /identity/v3/role_assignments?user.id=ef5dac3f311e43809a46995bab56e780&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 608 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:17.049418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dff40c-c158-4312-8b0b-a34b4e8b8fb4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:17.084392 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-2348809c-c3ed-4c67-a71a-728c956d177a None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:17.086609 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2348809c-c3ed-4c67-a71a-728c956d177a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:17.160720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:17.181919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85dff40c-c158-4312-8b0b-a34b4e8b8fb4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:17.203921 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 407/831] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:17.227206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348809c-c3ed-4c67-a71a-728c956d177a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:17.230984 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19b29c-e54f-4660-a3b2-b611705de3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:17.235104 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19b29c-e54f-4660-a3b2-b611705de3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:17.235399 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19b29c-e54f-4660-a3b2-b611705de3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:17.235807 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f19b29c-e54f-4660-a3b2-b611705de3a8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:17.244391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19b29c-e54f-4660-a3b2-b611705de3a8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:17.245859 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f19b29c-e54f-4660-a3b2-b611705de3a8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:17.263675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-6f19b29c-e54f-4660-a3b2-b611705de3a8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:17.263675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:17.263675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:17.263675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:17.263675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:17.263675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:17.263675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:17.263675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:17.263675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:17.263675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:17.263675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:17.263675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:17.263675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:54:17.263675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:54:17.263675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:54:17.264604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:54:17.264604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:17.264604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:17.264604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:17.264604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:17.264604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:17.264604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:17.264604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:17.264604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:17.264604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:17.264604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:17.264604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:17.264604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:17.264604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:17.264604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:17.264604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:17.264604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:17.265883 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:54:17.265883 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:54:17.265883 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:54:17.265883 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:54:17.265883 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:54:17.265883 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:54:17.265883 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:17.265883 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:17.265883 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 425/832] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:17.271131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19b29c-e54f-4660-a3b2-b611705de3a8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:17.334057 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:17.369587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2348809c-c3ed-4c67-a71a-728c956d177a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:17.404473 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f19b29c-e54f-4660-a3b2-b611705de3a8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:17.425200 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff46b01-d2c8-4260-b3dd-a485bef17909 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:17.426765 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff46b01-d2c8-4260-b3dd-a485bef17909 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:17.427560 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff46b01-d2c8-4260-b3dd-a485bef17909 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:17.427893 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dff46b01-d2c8-4260-b3dd-a485bef17909 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:17.442927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dff46b01-d2c8-4260-b3dd-a485bef17909 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:54:17.449605 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92050a-e434-44c9-b6ba-af41b1dc1f35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:17.450651 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92050a-e434-44c9-b6ba-af41b1dc1f35 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:17.451005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-dff46b01-d2c8-4260-b3dd-a485bef17909 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:54:17.451005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:17.451005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:17.451005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:17.451005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:17.451005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:17.451005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:17.451005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:17.451005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:17.451005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:17.451005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:17.451005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:17.451005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:54:17.451005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:54:17.451005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:54:17.451005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:54:17.452522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:54:17.452522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:54:17.452522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:54:17.452522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:54:17.452522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:17.452522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:17.452522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:17.452522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:17.452522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:17.452522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:17.452522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:17.452522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:17.452522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:17.452522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:17.452522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:17.452522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:17.453768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:17.453768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:17.453768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:17.453768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:17.453768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:54:17.453768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:54:17.453768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:54:17.453768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:54:17.453768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:54:17.453768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:54:17.453768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:54:17.453768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:17.455026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92050a-e434-44c9-b6ba-af41b1dc1f35 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:17.455520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b92050a-e434-44c9-b6ba-af41b1dc1f35 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:17.456555 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92050a-e434-44c9-b6ba-af41b1dc1f35 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:17.457075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92050a-e434-44c9-b6ba-af41b1dc1f35 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:17.458079 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 408/833] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:54:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:17.459083 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff46b01-d2c8-4260-b3dd-a485bef17909 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:17.459190 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b92050a-e434-44c9-b6ba-af41b1dc1f35 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:17.478955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 426/834] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:54:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:17.486948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92050a-e434-44c9-b6ba-af41b1dc1f35 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:17.511286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:17.606436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dff46b01-d2c8-4260-b3dd-a485bef17909 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:17.618506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b92050a-e434-44c9-b6ba-af41b1dc1f35 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:17.630733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:17.655573 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e9a1d-752c-4e98-ba51-808320e965e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:17.656313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e9a1d-752c-4e98-ba51-808320e965e0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:17.656609 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e9a1d-752c-4e98-ba51-808320e965e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:17.657103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386e9a1d-752c-4e98-ba51-808320e965e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:17.657692 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386e9a1d-752c-4e98-ba51-808320e965e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:17.658007 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386e9a1d-752c-4e98-ba51-808320e965e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:17.659463 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386e9a1d-752c-4e98-ba51-808320e965e0 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:17.667502 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50cbb5d9-9fa4-427f-a397-f6c363aafea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:17.668768 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbb5d9-9fa4-427f-a397-f6c363aafea9 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:17.669272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbb5d9-9fa4-427f-a397-f6c363aafea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:17.669771 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50cbb5d9-9fa4-427f-a397-f6c363aafea9 None admin] PATH_INFO: `/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/52076f59f68f4d77a0dadbf7a98d2b57/roles/1f7cf26664d04e5e9331ead293476442` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:17.674025 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 409/835] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:54:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:17.675254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e9a1d-752c-4e98-ba51-808320e965e0 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:17.690740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbb5d9-9fa4-427f-a397-f6c363aafea9 None admin] RBAC: Authorizing `identity:create_grant(project_id=08ee65f8b338491f80e7beac5e316b23, user_id=52076f59f68f4d77a0dadbf7a98d2b57, role_id=1f7cf26664d04e5e9331ead293476442)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:17.692733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50cbb5d9-9fa4-427f-a397-f6c363aafea9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:17.714358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 427/836] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:54:17 2022] PUT /identity/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/52076f59f68f4d77a0dadbf7a98d2b57/roles/1f7cf26664d04e5e9331ead293476442 => generated 0 bytes in 226 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:54:17.806744 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:17.816199 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386e9a1d-752c-4e98-ba51-808320e965e0 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:17.861300 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad9f14-b4ba-49cb-b4fa-9e6033153f3f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:17.862038 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad9f14-b4ba-49cb-b4fa-9e6033153f3f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:17.862379 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad9f14-b4ba-49cb-b4fa-9e6033153f3f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:17.862657 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad9f14-b4ba-49cb-b4fa-9e6033153f3f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:17.876834 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad9f14-b4ba-49cb-b4fa-9e6033153f3f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:17.878455 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad9f14-b4ba-49cb-b4fa-9e6033153f3f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:17.899358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-f6ad9f14-b4ba-49cb-b4fa-9e6033153f3f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:17.899358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:17.899358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:17.899358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:17.899358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:17.899358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:17.899358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:17.899358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:17.899358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:17.899358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:17.899358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:17.899358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:17.899358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:17.899358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:17.899358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:17.900319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:17.900319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:17.900319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:17.900319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:17.900319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:17.900319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:17.900319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:17.900319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:17.900319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:17.900319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:17.900319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:17.900319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:17.900319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:17.900319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:17.900319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:17.900319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:17.900319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:17.901201 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:17.901201 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:17.901201 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:17.901201 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:17.903405 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 410/837] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:17.905972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7752c-4523-4a28-a30d-c04412892ec6 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:17.906655 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7752c-4523-4a28-a30d-c04412892ec6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:17.907155 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c7752c-4523-4a28-a30d-c04412892ec6 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:17.910083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 428/838] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:17 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:17.912715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad9f14-b4ba-49cb-b4fa-9e6033153f3f None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:17.920397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed45e7e-379f-4d69-869c-ae9d8c6d532d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:17.920986 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed45e7e-379f-4d69-869c-ae9d8c6d532d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:17.921482 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ed45e7e-379f-4d69-869c-ae9d8c6d532d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:17.960256 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7ed45e7e-379f-4d69-869c-ae9d8c6d532d None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:17.963712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7ed45e7e-379f-4d69-869c-ae9d8c6d532d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:18.038496 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad9f14-b4ba-49cb-b4fa-9e6033153f3f None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:18.084556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 429/839] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:17 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:18.094241 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbba27-a069-4057-8d0c-cbb6183bfb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:18.094957 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbba27-a069-4057-8d0c-cbb6183bfb53 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:18.095237 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbba27-a069-4057-8d0c-cbb6183bfb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:18.098342 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92bbba27-a069-4057-8d0c-cbb6183bfb53 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:18.098342 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbba27-a069-4057-8d0c-cbb6183bfb53 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:18.098342 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbba27-a069-4057-8d0c-cbb6183bfb53 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:18.098342 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92bbba27-a069-4057-8d0c-cbb6183bfb53 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:18.108040 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed45e7e-379f-4d69-869c-ae9d8c6d532d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:18.114062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 411/840] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:18.122336 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbba27-a069-4057-8d0c-cbb6183bfb53 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:18.131151 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:18.224880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:18.239455 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92bbba27-a069-4057-8d0c-cbb6183bfb53 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:18.242442 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ed45e7e-379f-4d69-869c-ae9d8c6d532d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:18.277300 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7030a-5da4-4d2d-af63-301bf35fcdd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:18.278284 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7030a-5da4-4d2d-af63-301bf35fcdd9 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:18.278599 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7030a-5da4-4d2d-af63-301bf35fcdd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:18.278878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b7030a-5da4-4d2d-af63-301bf35fcdd9 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:18.281575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6e7f4-ed93-4d71-b923-1ee3fedd87c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:18.282674 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6e7f4-ed93-4d71-b923-1ee3fedd87c9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:18.283158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6e7f4-ed93-4d71-b923-1ee3fedd87c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:18.283626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c6e7f4-ed93-4d71-b923-1ee3fedd87c9 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:18.284503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6e7f4-ed93-4d71-b923-1ee3fedd87c9 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:18.286105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c6e7f4-ed93-4d71-b923-1ee3fedd87c9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:18.288007 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7030a-5da4-4d2d-af63-301bf35fcdd9 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:18.289740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b7030a-5da4-4d2d-af63-301bf35fcdd9 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:18.300083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-38c6e7f4-ed93-4d71-b923-1ee3fedd87c9 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 13:54:18.300083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:18.300083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:18.300083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:18.300083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:18.300083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:18.300083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:18.300083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:18.300083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 13:54:18.300083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:18.300083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:18.300083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:18.300083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:18.300083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:18.300083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:18.300083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:18.300083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:18.301226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:18.301226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:18.301226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:18.301226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:18.301226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:18.301226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:18.301226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:18.301226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:18.301226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:18.301226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:18.301226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:18.301226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:18.301226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:18.301226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:18.301226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:18.301226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:18.302058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:18.302058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:18.302058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:18.302058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:18.302058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:18.302058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:18.302058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:18.302058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:18.302058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:18.302058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:18.302058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:18.302058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:18.302058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 13:54:18.302058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:18.305412 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 430/841] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:54:18 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:18.305538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-d4b7030a-5da4-4d2d-af63-301bf35fcdd9 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 11 13:54:18.305538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:18.305538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:18.305538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:18.305538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:18.305538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:18.305538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:18.305538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:18.305538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:18.305538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:18.305538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:18.305538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:18.305538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:54:18.305538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:54:18.305538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:54:18.305538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:54:18.306677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:18.306677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:18.306677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:54:18.306677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:18.306677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:54:18.306677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:18.306677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:18.306677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:18.306677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:18.306677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:18.306677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:18.306677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:18.306677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:18.306677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:18.306677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:18.306677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:18.306677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:18.306677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:18.307594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:18.307594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:18.307594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:54:18.307594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:54:18.307594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:54:18.307594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:54:18.307594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:54:18.307594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:54:18.307594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 11 13:54:18.307594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:18.311018 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6e7f4-ed93-4d71-b923-1ee3fedd87c9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:18.311555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 412/842] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:54:18 2022] GET /identity/v3/users/swift => generated 83 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:18.312494 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7030a-5da4-4d2d-af63-301bf35fcdd9 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:18.387321 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:18.426198 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b7030a-5da4-4d2d-af63-301bf35fcdd9 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:18.449721 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c6e7f4-ed93-4d71-b923-1ee3fedd87c9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:18.463579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9b6a0-1d7b-43bc-90e6-da4bf5a305dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:18.464265 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9b6a0-1d7b-43bc-90e6-da4bf5a305dd None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:18.464550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9b6a0-1d7b-43bc-90e6-da4bf5a305dd None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:18.464828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e9b6a0-1d7b-43bc-90e6-da4bf5a305dd None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:18.465684 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9b6a0-1d7b-43bc-90e6-da4bf5a305dd None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:18.465997 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9b6a0-1d7b-43bc-90e6-da4bf5a305dd None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:18.468292 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e9b6a0-1d7b-43bc-90e6-da4bf5a305dd None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:18.492196 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 413/843] 10.222.0.49 () {56 vars in 1135 bytes} [Mon Jul 11 13:54:18 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:18.493106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9b6a0-1d7b-43bc-90e6-da4bf5a305dd None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:18.499091 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce21678-294c-464d-a536-929691a208bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:18.500673 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce21678-294c-464d-a536-929691a208bc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:18.501441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce21678-294c-464d-a536-929691a208bc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:18.502211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce21678-294c-464d-a536-929691a208bc None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:18.503621 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce21678-294c-464d-a536-929691a208bc None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:18.504474 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce21678-294c-464d-a536-929691a208bc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:18.507428 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ce21678-294c-464d-a536-929691a208bc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:18.517571 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 431/844] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:54:18 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:18.526063 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce21678-294c-464d-a536-929691a208bc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:18.543667 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:18.603932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e9b6a0-1d7b-43bc-90e6-da4bf5a305dd None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:18.631878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:18.637303 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce21678-294c-464d-a536-929691a208bc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:18.643496 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd837ef8-1194-4ba4-a1fe-554d5ce2ca28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:18.644086 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd837ef8-1194-4ba4-a1fe-554d5ce2ca28 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:18.644369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd837ef8-1194-4ba4-a1fe-554d5ce2ca28 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:18.644643 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd837ef8-1194-4ba4-a1fe-554d5ce2ca28 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:18.675981 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc138e90-984f-4e47-89ee-03a96a0a997f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:18.677005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc138e90-984f-4e47-89ee-03a96a0a997f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:18.677491 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc138e90-984f-4e47-89ee-03a96a0a997f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:18.677963 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc138e90-984f-4e47-89ee-03a96a0a997f None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:18.678949 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc138e90-984f-4e47-89ee-03a96a0a997f None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:18.679453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc138e90-984f-4e47-89ee-03a96a0a997f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:18.681298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc138e90-984f-4e47-89ee-03a96a0a997f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:18.694623 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 432/845] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:54:18 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:18.724180 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd837ef8-1194-4ba4-a1fe-554d5ce2ca28 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:18.725688 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd837ef8-1194-4ba4-a1fe-554d5ce2ca28 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:18.848868 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 414/846] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:54:18 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 13:54:18.854678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc138e90-984f-4e47-89ee-03a96a0a997f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:18.963236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:18.986266 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc138e90-984f-4e47-89ee-03a96a0a997f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:19.027515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8744f-08a0-487d-96a9-6605a0c3ae1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:19.028713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8744f-08a0-487d-96a9-6605a0c3ae1f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:19.029751 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8744f-08a0-487d-96a9-6605a0c3ae1f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:19.030234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad8744f-08a0-487d-96a9-6605a0c3ae1f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:19.042552 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8744f-08a0-487d-96a9-6605a0c3ae1f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:19.044355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad8744f-08a0-487d-96a9-6605a0c3ae1f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:19.063348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-4ad8744f-08a0-487d-96a9-6605a0c3ae1f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:19.063348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:19.063348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:19.063348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:19.063348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:19.063348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:19.063348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:19.063348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:19.063348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:19.063348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:19.063348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:19.063348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:19.063348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:19.063348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:19.063348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:19.064301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:19.064301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:19.064301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:19.064301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:19.064301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:19.064301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:19.064301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:19.064301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:19.064301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:19.064301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:19.064301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:19.064301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:19.064301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:19.064301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:19.064301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:19.064301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:19.064301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:19.065181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:19.065181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:19.065181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:19.065181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:19.067511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 433/847] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:18 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:19.078025 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd837ef8-1194-4ba4-a1fe-554d5ce2ca28 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:19.129661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:19.216180 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd837ef8-1194-4ba4-a1fe-554d5ce2ca28 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:19.256571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96e32f9-44fc-4d71-a014-ef38940353af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:19.257255 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e32f9-44fc-4d71-a014-ef38940353af None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:19.257542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e32f9-44fc-4d71-a014-ef38940353af None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:19.257823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96e32f9-44fc-4d71-a014-ef38940353af None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:19.258593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e32f9-44fc-4d71-a014-ef38940353af None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:19.258904 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e32f9-44fc-4d71-a014-ef38940353af None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:19.260299 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96e32f9-44fc-4d71-a014-ef38940353af None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:19.273546 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 415/848] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:19.279311 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8744f-08a0-487d-96a9-6605a0c3ae1f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:19.289625 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8862d-b233-486c-94a9-6b66a4a2953e None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:19.289939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8862d-b233-486c-94a9-6b66a4a2953e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:19.290199 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6df8862d-b233-486c-94a9-6b66a4a2953e None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:19.291921 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 416/849] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:54:19.298147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e99f0-76c1-4aaa-9510-66b6b47db9a7 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:19.298484 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e99f0-76c1-4aaa-9510-66b6b47db9a7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:19.298839 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-479e99f0-76c1-4aaa-9510-66b6b47db9a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:19.316356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-479e99f0-76c1-4aaa-9510-66b6b47db9a7 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:19.317788 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-479e99f0-76c1-4aaa-9510-66b6b47db9a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:19.409395 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 417/850] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:19 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:54:19.415699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad8744f-08a0-487d-96a9-6605a0c3ae1f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:19.435908 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479e99f0-76c1-4aaa-9510-66b6b47db9a7 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:19.456452 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:19.466523 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c88402-13b6-4da4-a5de-b140690b45b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:19.468565 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c88402-13b6-4da4-a5de-b140690b45b7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:19.469197 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c88402-13b6-4da4-a5de-b140690b45b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:19.469704 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c88402-13b6-4da4-a5de-b140690b45b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:19.471964 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:19.489700 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c88402-13b6-4da4-a5de-b140690b45b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:19.491998 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c88402-13b6-4da4-a5de-b140690b45b7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:19.523889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-20c88402-13b6-4da4-a5de-b140690b45b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:19.523889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:19.523889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:19.523889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:19.523889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:19.523889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:19.523889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:19.523889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:19.523889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:19.523889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:19.523889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:19.523889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:19.523889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:54:19.523889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:54:19.523889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:54:19.525182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:54:19.525182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:19.525182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:19.525182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:19.525182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:19.525182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:19.525182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:19.525182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:19.525182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:19.525182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:19.525182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:19.525182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:19.525182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:19.525182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:19.525182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:19.525182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:19.525182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:19.526236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:54:19.526236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:54:19.526236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:54:19.526236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:54:19.526236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:54:19.526236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:54:19.526236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:19.526236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:19.532772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 434/851] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:19.535560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c88402-13b6-4da4-a5de-b140690b45b7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:19.587850 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-479e99f0-76c1-4aaa-9510-66b6b47db9a7 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:19.625790 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8237d6-b374-4f5e-b954-4b54a2fb7161 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:19.627034 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8237d6-b374-4f5e-b954-4b54a2fb7161 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:19.627571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8237d6-b374-4f5e-b954-4b54a2fb7161 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:19.628109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8237d6-b374-4f5e-b954-4b54a2fb7161 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:19.628986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8237d6-b374-4f5e-b954-4b54a2fb7161 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:19.631957 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd8237d6-b374-4f5e-b954-4b54a2fb7161 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:19.646076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-cd8237d6-b374-4f5e-b954-4b54a2fb7161 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 13:54:19.646076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:19.646076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:19.646076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:19.646076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:19.646076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:19.646076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:19.646076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:19.646076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 13:54:19.646076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:19.646076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:19.646076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:19.646076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:19.646076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:19.646076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:19.646076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:19.647123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:19.647123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:19.647123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:19.647123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:19.647123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:19.647123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:19.647123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:19.647123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:19.647123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:19.647123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:19.647123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:19.647123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:19.647123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:19.647123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:19.647123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:19.647123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:19.647966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:19.647966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:19.647966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:19.647966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:19.647966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:19.647966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:19.647966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:19.647966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:19.647966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:19.647966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:19.647966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:19.647966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:19.647966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:19.647966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 13:54:19.647966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:19.649331 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c88402-13b6-4da4-a5de-b140690b45b7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:19.651155 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 418/852] 10.222.0.49 () {56 vars in 1132 bytes} [Mon Jul 11 13:54:19 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:19.657005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8237d6-b374-4f5e-b954-4b54a2fb7161 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:19.692892 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b5075-df9a-4e2f-8f69-9bc557d89a32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:19.693615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b5075-df9a-4e2f-8f69-9bc557d89a32 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:19.693901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b5075-df9a-4e2f-8f69-9bc557d89a32 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:19.694229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b87b5075-df9a-4e2f-8f69-9bc557d89a32 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:19.695107 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b5075-df9a-4e2f-8f69-9bc557d89a32 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:19.695377 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b5075-df9a-4e2f-8f69-9bc557d89a32 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:19.696815 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b87b5075-df9a-4e2f-8f69-9bc557d89a32 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:19.716339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 435/853] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:54:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:19.717970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b5075-df9a-4e2f-8f69-9bc557d89a32 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:19.759340 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:19.784123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8237d6-b374-4f5e-b954-4b54a2fb7161 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:19.802972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:19.824275 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6720a54-9659-4a39-931f-edc723352934 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:19.825214 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6720a54-9659-4a39-931f-edc723352934 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:19.825652 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6720a54-9659-4a39-931f-edc723352934 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:19.826667 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6720a54-9659-4a39-931f-edc723352934 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:19.827549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6720a54-9659-4a39-931f-edc723352934 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:19.828013 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6720a54-9659-4a39-931f-edc723352934 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:19.829604 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6720a54-9659-4a39-931f-edc723352934 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:19.835474 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 419/854] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 11 13:54:19 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:19.841045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6720a54-9659-4a39-931f-edc723352934 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:19.873658 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b87b5075-df9a-4e2f-8f69-9bc557d89a32 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:19.920450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a525f5-9af8-4c8e-8efa-da7f91eacb55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:19.921694 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a525f5-9af8-4c8e-8efa-da7f91eacb55 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:19.922391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a525f5-9af8-4c8e-8efa-da7f91eacb55 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:19.923045 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a525f5-9af8-4c8e-8efa-da7f91eacb55 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:19.924653 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a525f5-9af8-4c8e-8efa-da7f91eacb55 None admin] RBAC: Adding query filter params (role.id=6964639a7f6c41e8a6701ef533151c41, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=d9c1369033a44c1c81d4c1c9ff2bbedb) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:19.925343 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a525f5-9af8-4c8e-8efa-da7f91eacb55 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:19.927555 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a525f5-9af8-4c8e-8efa-da7f91eacb55 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:19.937334 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 436/855] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:54:19 2022] GET /identity/v3/role_assignments?user.id=d9c1369033a44c1c81d4c1c9ff2bbedb&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 608 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:19.974497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6720a54-9659-4a39-931f-edc723352934 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:20.022616 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16b504-e91f-4247-ae80-e52ec64d3d30 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:20.022915 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16b504-e91f-4247-ae80-e52ec64d3d30 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:20.023193 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae16b504-e91f-4247-ae80-e52ec64d3d30 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:20.025031 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e4bc5-fafe-454f-ab9a-589317063d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:20.026005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e4bc5-fafe-454f-ab9a-589317063d93 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:20.026666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e4bc5-fafe-454f-ab9a-589317063d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:20.027349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-504e4bc5-fafe-454f-ab9a-589317063d93 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:20.028412 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e4bc5-fafe-454f-ab9a-589317063d93 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:20.028900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 437/856] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:54:20.029407 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e4bc5-fafe-454f-ab9a-589317063d93 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:20.030983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f2acc-2fd6-4e7e-a773-eadd54b886be None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:20.031276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f2acc-2fd6-4e7e-a773-eadd54b886be None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:20.031548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940f2acc-2fd6-4e7e-a773-eadd54b886be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:20.032261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-504e4bc5-fafe-454f-ab9a-589317063d93 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:20.040245 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 420/857] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 11 13:54:19 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:20.057495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-940f2acc-2fd6-4e7e-a773-eadd54b886be None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:20.058816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-940f2acc-2fd6-4e7e-a773-eadd54b886be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:20.161710 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 438/858] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:20 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:54:20.184813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e4bc5-fafe-454f-ab9a-589317063d93 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:20.201133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:20.334390 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-504e4bc5-fafe-454f-ab9a-589317063d93 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:20.397442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024550f-cbd9-4a82-8cf4-51c4c1d2f579 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:20.398712 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024550f-cbd9-4a82-8cf4-51c4c1d2f579 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:20.399230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024550f-cbd9-4a82-8cf4-51c4c1d2f579 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:20.399772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3024550f-cbd9-4a82-8cf4-51c4c1d2f579 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:20.418879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3024550f-cbd9-4a82-8cf4-51c4c1d2f579 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:54:20.432580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-3024550f-cbd9-4a82-8cf4-51c4c1d2f579 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:54:20.432580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:20.432580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:20.432580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:20.432580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:20.432580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:20.432580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:20.432580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:20.432580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:20.432580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:20.432580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:20.432580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:20.432580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:54:20.432580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:54:20.432580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:54:20.432580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:54:20.433630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:54:20.433630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:54:20.433630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:54:20.433630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:54:20.433630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:20.433630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:20.433630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:20.433630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:20.433630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:20.433630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:20.433630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:20.433630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:20.433630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:20.433630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:20.433630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:20.433630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:20.434823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:20.434823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:20.434823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:20.434823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:20.434823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:54:20.434823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:54:20.434823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:54:20.434823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:54:20.434823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:54:20.434823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:54:20.434823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:54:20.434823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:20.434823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 421/859] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:54:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:20.439595 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f2acc-2fd6-4e7e-a773-eadd54b886be None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:20.506161 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:20.570581 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940f2acc-2fd6-4e7e-a773-eadd54b886be None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:20.613197 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d4f9e-698f-46bc-867f-8acb3eb9792a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:20.613867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d4f9e-698f-46bc-867f-8acb3eb9792a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:20.614205 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d4f9e-698f-46bc-867f-8acb3eb9792a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:20.614561 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea4d4f9e-698f-46bc-867f-8acb3eb9792a None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:20.615381 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d4f9e-698f-46bc-867f-8acb3eb9792a None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:20.615692 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d4f9e-698f-46bc-867f-8acb3eb9792a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:20.617225 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea4d4f9e-698f-46bc-867f-8acb3eb9792a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:20.627424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 439/860] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:54:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:20.634799 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024550f-cbd9-4a82-8cf4-51c4c1d2f579 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:20.743840 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:20.815352 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3024550f-cbd9-4a82-8cf4-51c4c1d2f579 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:20.878454 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed1429c1-f806-424e-b858-0afd6f5fa6b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:20.879707 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1429c1-f806-424e-b858-0afd6f5fa6b0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:20.880184 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1429c1-f806-424e-b858-0afd6f5fa6b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:20.880749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed1429c1-f806-424e-b858-0afd6f5fa6b0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:20.899377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1429c1-f806-424e-b858-0afd6f5fa6b0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:20.901685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed1429c1-f806-424e-b858-0afd6f5fa6b0 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:20.932622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-ed1429c1-f806-424e-b858-0afd6f5fa6b0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:20.932622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:20.932622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:20.932622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:20.932622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:20.932622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:20.932622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:20.932622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:20.932622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:20.932622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:20.932622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:20.932622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:20.932622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:20.932622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:20.932622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:20.934882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:20.934882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:20.934882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:20.934882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:20.934882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:20.934882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:20.934882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:20.934882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:20.934882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:20.934882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:20.934882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:20.934882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:20.934882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:20.934882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:20.934882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:20.934882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:20.934882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:20.937520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:20.937520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:20.937520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:20.937520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:20.937520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 422/861] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:20.943801 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d4f9e-698f-46bc-867f-8acb3eb9792a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:21.013810 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403d3d2-8d01-4316-b8df-ecbc4b95e67f None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:21.014499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403d3d2-8d01-4316-b8df-ecbc4b95e67f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:21.014980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6403d3d2-8d01-4316-b8df-ecbc4b95e67f None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:21.021042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 423/862] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:21 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:54:21.022656 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e16d8b-1d33-45fe-873d-5e115eae93fe None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:21.023149 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e16d8b-1d33-45fe-873d-5e115eae93fe None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:21.023731 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e16d8b-1d33-45fe-873d-5e115eae93fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:21.041576 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-34e16d8b-1d33-45fe-873d-5e115eae93fe None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:21.043153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-34e16d8b-1d33-45fe-873d-5e115eae93fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:21.052900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea4d4f9e-698f-46bc-867f-8acb3eb9792a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:21.066720 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:21.095623 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93542578-d6fe-47b4-8cb2-046d29b168f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:21.096461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93542578-d6fe-47b4-8cb2-046d29b168f3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:21.096858 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93542578-d6fe-47b4-8cb2-046d29b168f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:21.097241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93542578-d6fe-47b4-8cb2-046d29b168f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:21.098089 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93542578-d6fe-47b4-8cb2-046d29b168f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:21.098528 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93542578-d6fe-47b4-8cb2-046d29b168f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:21.100001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93542578-d6fe-47b4-8cb2-046d29b168f3 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:21.117321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 440/863] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:21.118619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93542578-d6fe-47b4-8cb2-046d29b168f3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:21.138459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 424/864] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:21 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:21.166813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e16d8b-1d33-45fe-873d-5e115eae93fe None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:21.254941 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:21.319842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93542578-d6fe-47b4-8cb2-046d29b168f3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:21.359003 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e16d8b-1d33-45fe-873d-5e115eae93fe None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:21.391041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027c16c-16e7-4675-aeda-41d031da81a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:21.392506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027c16c-16e7-4675-aeda-41d031da81a0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:21.393028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027c16c-16e7-4675-aeda-41d031da81a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:21.393549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2027c16c-16e7-4675-aeda-41d031da81a0 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:21.409445 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e723967-c3df-426a-85d4-59ba0c67d692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:21.409694 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027c16c-16e7-4675-aeda-41d031da81a0 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:21.410248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e723967-c3df-426a-85d4-59ba0c67d692 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:21.413607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e723967-c3df-426a-85d4-59ba0c67d692 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:21.413607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e723967-c3df-426a-85d4-59ba0c67d692 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:21.413951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2027c16c-16e7-4675-aeda-41d031da81a0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:21.414733 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e723967-c3df-426a-85d4-59ba0c67d692 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:21.416222 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e723967-c3df-426a-85d4-59ba0c67d692 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:21.434178 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 425/865] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 11 13:54:21 2022] POST /identity/v3/services => generated 251 bytes in 278 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 13:54:21.445173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-2027c16c-16e7-4675-aeda-41d031da81a0 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 13:54:21.445173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:21.445173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:21.445173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:21.445173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:21.445173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:21.445173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:21.445173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:21.445173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:21.445173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:21.445173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:21.445173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:21.445173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:54:21.445173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:54:21.445173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:54:21.445173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:54:21.447920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:21.447920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:21.447920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:54:21.447920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:21.447920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:54:21.447920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:21.447920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:21.447920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:21.447920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:21.447920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:21.447920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:21.447920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:21.447920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:21.447920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:21.447920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:21.447920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:21.447920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:21.447920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:21.450486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:21.450486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:21.450486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:54:21.450486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:54:21.450486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:54:21.450486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:54:21.450486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:54:21.450486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:54:21.450486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 13:54:21.450486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:21.450486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 441/866] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 11 13:54:21 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:21.463900 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e723967-c3df-426a-85d4-59ba0c67d692 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:21.648926 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e723967-c3df-426a-85d4-59ba0c67d692 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:21.662196 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:21.697507 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08815f5-fb17-4361-86a4-fb5d199289ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:21.698628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08815f5-fb17-4361-86a4-fb5d199289ec None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:21.699177 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08815f5-fb17-4361-86a4-fb5d199289ec None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:21.699672 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08815f5-fb17-4361-86a4-fb5d199289ec None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:21.700773 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08815f5-fb17-4361-86a4-fb5d199289ec None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:21.701363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08815f5-fb17-4361-86a4-fb5d199289ec None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:21.703158 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b08815f5-fb17-4361-86a4-fb5d199289ec None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:21.724467 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 426/867] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Jul 11 13:54:21 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:21.730810 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027c16c-16e7-4675-aeda-41d031da81a0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:21.768174 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:21.869730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2027c16c-16e7-4675-aeda-41d031da81a0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:21.921696 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd53f85-a93b-48e0-809c-f3e3661c5e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:21.922690 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd53f85-a93b-48e0-809c-f3e3661c5e6c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:21.923010 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd53f85-a93b-48e0-809c-f3e3661c5e6c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:21.923298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd53f85-a93b-48e0-809c-f3e3661c5e6c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:22.027499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd53f85-a93b-48e0-809c-f3e3661c5e6c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:22.031284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcd53f85-a93b-48e0-809c-f3e3661c5e6c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:22.089806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:22.245499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 442/868] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:54:21 2022] GET /identity/v3/auth/tokens => generated 1252 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 13:54:22.251079 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08815f5-fb17-4361-86a4-fb5d199289ec None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:22.366692 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08815f5-fb17-4361-86a4-fb5d199289ec None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:22.405553 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e679dd6f-b8f1-45bf-a356-60282a26fdeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:22.406387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679dd6f-b8f1-45bf-a356-60282a26fdeb None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:22.406675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679dd6f-b8f1-45bf-a356-60282a26fdeb None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:22.406952 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e679dd6f-b8f1-45bf-a356-60282a26fdeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:22.418390 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679dd6f-b8f1-45bf-a356-60282a26fdeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:22.420412 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e679dd6f-b8f1-45bf-a356-60282a26fdeb None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:22.439794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-e679dd6f-b8f1-45bf-a356-60282a26fdeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:22.439794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:22.439794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:22.439794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:22.439794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:22.439794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:22.439794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:22.439794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:22.439794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:22.439794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:22.439794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:22.439794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:22.439794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:22.439794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:22.439794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:22.440772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:22.440772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:22.440772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:22.440772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:22.440772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:22.440772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:22.440772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:22.440772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:22.440772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:22.440772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:22.440772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:22.440772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:22.440772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:22.440772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:22.440772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:22.440772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:22.440772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:22.442686 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:22.442686 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:22.442686 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:22.442686 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:22.442686 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 427/869] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:22.447734 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd53f85-a93b-48e0-809c-f3e3661c5e6c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:22.460889 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53885420-e808-4551-92d7-a2a0eb0416ca None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:22.461618 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53885420-e808-4551-92d7-a2a0eb0416ca None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:22.462127 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53885420-e808-4551-92d7-a2a0eb0416ca None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:22.464461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 428/870] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:54:22.468309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14b530-b6c9-4a3d-9c1b-95d7aa128f57 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:22.469298 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14b530-b6c9-4a3d-9c1b-95d7aa128f57 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:22.469587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f14b530-b6c9-4a3d-9c1b-95d7aa128f57 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:22.471279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 429/871] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:22 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:22.472128 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f6bf2-6da1-4865-b475-88149c810f3f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:22.472415 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f6bf2-6da1-4865-b475-88149c810f3f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:22.472689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8f6bf2-6da1-4865-b475-88149c810f3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:22.491775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9a8f6bf2-6da1-4865-b475-88149c810f3f None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:22.492991 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9a8f6bf2-6da1-4865-b475-88149c810f3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:22.512757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:22.582293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 430/872] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:22 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:22.584187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac4e33-8d30-4a08-932c-c1baa063721b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:22.584715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac4e33-8d30-4a08-932c-c1baa063721b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:22.585195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bac4e33-8d30-4a08-932c-c1baa063721b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:22.606785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd53f85-a93b-48e0-809c-f3e3661c5e6c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:22.615680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5bac4e33-8d30-4a08-932c-c1baa063721b None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:22.617891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5bac4e33-8d30-4a08-932c-c1baa063721b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:22.641233 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:22.650675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea9b4f-79c6-45ef-8d0c-8f00a7501c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:22.651479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea9b4f-79c6-45ef-8d0c-8f00a7501c62 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:22.651811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea9b4f-79c6-45ef-8d0c-8f00a7501c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:22.652093 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ea9b4f-79c6-45ef-8d0c-8f00a7501c62 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:22.652979 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea9b4f-79c6-45ef-8d0c-8f00a7501c62 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:22.653377 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea9b4f-79c6-45ef-8d0c-8f00a7501c62 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:22.655356 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ea9b4f-79c6-45ef-8d0c-8f00a7501c62 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:22.675627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 443/873] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 224 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:22.675627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea9b4f-79c6-45ef-8d0c-8f00a7501c62 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:22.707371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 431/874] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:22 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:22.708450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac4e33-8d30-4a08-932c-c1baa063721b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:22.803898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:22.817043 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ea9b4f-79c6-45ef-8d0c-8f00a7501c62 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:22.835826 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bac4e33-8d30-4a08-932c-c1baa063721b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:22.860039 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12cbcd-c867-4fbf-9f63-2c50d0179662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:22.860726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12cbcd-c867-4fbf-9f63-2c50d0179662 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:22.861009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12cbcd-c867-4fbf-9f63-2c50d0179662 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:22.861290 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca12cbcd-c867-4fbf-9f63-2c50d0179662 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:22.862034 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12cbcd-c867-4fbf-9f63-2c50d0179662 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:22.863478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca12cbcd-c867-4fbf-9f63-2c50d0179662 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:22.879983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 444/875] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 11 13:54:22 2022] POST /identity/v3/services => generated 269 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:54:22.880854 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12cbcd-c867-4fbf-9f63-2c50d0179662 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:22.911226 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5feff-aa00-49d8-b63b-e2893b28f83d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:22.913027 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5feff-aa00-49d8-b63b-e2893b28f83d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:22.913991 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5feff-aa00-49d8-b63b-e2893b28f83d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:22.914847 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f5feff-aa00-49d8-b63b-e2893b28f83d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:22.931012 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5feff-aa00-49d8-b63b-e2893b28f83d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:22.934440 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0f5feff-aa00-49d8-b63b-e2893b28f83d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:22.957294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:22.968375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-c0f5feff-aa00-49d8-b63b-e2893b28f83d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:22.968375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:22.968375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:22.968375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:22.968375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:22.968375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:22.968375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:22.968375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:22.968375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:22.968375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:22.968375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:22.968375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:22.968375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:54:22.968375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:54:22.968375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:54:22.970210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:54:22.970210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:22.970210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:22.970210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:22.970210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:22.970210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:22.970210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:22.970210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:22.970210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:22.970210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:22.970210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:22.970210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:22.970210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:22.970210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:22.970210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:22.970210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:22.970210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:22.972078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:54:22.972078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:54:22.972078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:54:22.972078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:54:22.972078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:54:22.972078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:54:22.972078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:22.972078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:22.977782 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 432/876] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 270 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:22.985972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5feff-aa00-49d8-b63b-e2893b28f83d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:23.048788 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca12cbcd-c867-4fbf-9f63-2c50d0179662 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:23.106410 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910105e6-6592-4912-a1f2-16c467fbfe20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:23.107785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910105e6-6592-4912-a1f2-16c467fbfe20 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:23.108111 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910105e6-6592-4912-a1f2-16c467fbfe20 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:23.108348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910105e6-6592-4912-a1f2-16c467fbfe20 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:23.109029 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910105e6-6592-4912-a1f2-16c467fbfe20 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:23.109526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f5feff-aa00-49d8-b63b-e2893b28f83d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:23.110473 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-910105e6-6592-4912-a1f2-16c467fbfe20 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:23.124240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-910105e6-6592-4912-a1f2-16c467fbfe20 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 13:54:23.124240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:23.124240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:23.124240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:23.124240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:23.124240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:23.124240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:23.124240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:23.124240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 13:54:23.124240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:23.124240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:23.124240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:23.124240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:23.124240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:23.124240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:23.124240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:23.124240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:23.125313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:23.125313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:23.125313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:23.125313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:23.125313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:23.125313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:23.125313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:23.125313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:23.125313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:23.125313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:23.125313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:23.125313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:23.125313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:23.125313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:23.125313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:23.125313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:23.126150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:23.126150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:23.126150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:23.126150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:23.126150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:23.126150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:23.126150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:23.126150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:23.126150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:23.126150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:23.126150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:23.126150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:23.126150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 13:54:23.126150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:23.126150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 445/877] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 11 13:54:22 2022] GET /identity/v3/services/object-store => generated 93 bytes in 245 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:23.133408 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910105e6-6592-4912-a1f2-16c467fbfe20 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:23.146911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:23.155204 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff6b0f-ce6b-4613-afd3-99527d46769c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:23.156075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff6b0f-ce6b-4613-afd3-99527d46769c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:23.156791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff6b0f-ce6b-4613-afd3-99527d46769c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:23.157177 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ff6b0f-ce6b-4613-afd3-99527d46769c None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:23.158598 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ff6b0f-ce6b-4613-afd3-99527d46769c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:23.159058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ff6b0f-ce6b-4613-afd3-99527d46769c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:23.161327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ff6b0f-ce6b-4613-afd3-99527d46769c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:23.180734 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 433/878] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:54:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:23.187920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff6b0f-ce6b-4613-afd3-99527d46769c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:23.284725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910105e6-6592-4912-a1f2-16c467fbfe20 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:23.307163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ff6b0f-ce6b-4613-afd3-99527d46769c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:23.307821 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:23.325275 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a87eb-5a3c-48fd-a7ca-aeb6cd2d3d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:23.326191 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a87eb-5a3c-48fd-a7ca-aeb6cd2d3d04 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:23.326462 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a87eb-5a3c-48fd-a7ca-aeb6cd2d3d04 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:23.327270 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f03a87eb-5a3c-48fd-a7ca-aeb6cd2d3d04 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:23.328150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a87eb-5a3c-48fd-a7ca-aeb6cd2d3d04 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:23.328579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a87eb-5a3c-48fd-a7ca-aeb6cd2d3d04 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:23.330358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f03a87eb-5a3c-48fd-a7ca-aeb6cd2d3d04 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:23.340939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 446/879] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 11 13:54:23 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:23.342786 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a87eb-5a3c-48fd-a7ca-aeb6cd2d3d04 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:23.352567 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16370205-2f3d-4099-b200-ec87335f7390 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:23.353354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16370205-2f3d-4099-b200-ec87335f7390 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:23.353644 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16370205-2f3d-4099-b200-ec87335f7390 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:23.354035 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16370205-2f3d-4099-b200-ec87335f7390 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:23.354973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16370205-2f3d-4099-b200-ec87335f7390 None admin] RBAC: Adding query filter params (role.id=1f7cf26664d04e5e9331ead293476442, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=52076f59f68f4d77a0dadbf7a98d2b57) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:23.355297 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16370205-2f3d-4099-b200-ec87335f7390 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:23.357400 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16370205-2f3d-4099-b200-ec87335f7390 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:23.365168 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 434/880] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:54:23 2022] GET /identity/v3/role_assignments?user.id=52076f59f68f4d77a0dadbf7a98d2b57&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=1f7cf26664d04e5e9331ead293476442 => generated 608 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:23.478956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f03a87eb-5a3c-48fd-a7ca-aeb6cd2d3d04 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:23.530191 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264e711d-fe22-49b2-b220-4350caa86825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:23.531215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e711d-fe22-49b2-b220-4350caa86825 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:23.531678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e711d-fe22-49b2-b220-4350caa86825 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:23.532186 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264e711d-fe22-49b2-b220-4350caa86825 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:23.533148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e711d-fe22-49b2-b220-4350caa86825 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:23.533634 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e711d-fe22-49b2-b220-4350caa86825 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:23.535403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-264e711d-fe22-49b2-b220-4350caa86825 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:23.546735 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 447/881] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 11 13:54:23 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:23.919201 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d18a53-5c4f-4916-882a-c90eaa23b9dc None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:23.919809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d18a53-5c4f-4916-882a-c90eaa23b9dc None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:23.920186 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68d18a53-5c4f-4916-882a-c90eaa23b9dc None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:23.923259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 435/882] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:23 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:23.932770 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05d020-854c-4d5d-9011-50f5cf1befd9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:23.933298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05d020-854c-4d5d-9011-50f5cf1befd9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:23.933808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a05d020-854c-4d5d-9011-50f5cf1befd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:23.963410 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2a05d020-854c-4d5d-9011-50f5cf1befd9 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:23.965508 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2a05d020-854c-4d5d-9011-50f5cf1befd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:24.012440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:24.051646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 448/883] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:23 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:24.075271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d18a53-5c4f-4916-882a-c90eaa23b9dc None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:24.153809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:24.207048 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68d18a53-5c4f-4916-882a-c90eaa23b9dc None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:24.246533 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37db91-ee71-4a34-9101-99656bca2b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:24.247363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37db91-ee71-4a34-9101-99656bca2b17 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:24.247648 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37db91-ee71-4a34-9101-99656bca2b17 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:24.248463 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad37db91-ee71-4a34-9101-99656bca2b17 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:24.249148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37db91-ee71-4a34-9101-99656bca2b17 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:24.250604 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad37db91-ee71-4a34-9101-99656bca2b17 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:24.264375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-ad37db91-ee71-4a34-9101-99656bca2b17 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 13:54:24.264375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:24.264375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:24.264375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:24.264375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:24.264375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:24.264375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:24.264375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:24.264375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 13:54:24.264375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:24.264375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:24.264375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:24.264375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:24.264375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:24.264375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:24.264375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:24.264375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:24.265434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:24.265434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:24.265434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:24.265434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:24.265434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:24.265434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:24.265434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:24.265434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:24.265434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:24.265434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:24.265434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:24.265434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:24.265434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:24.265434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:24.265434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:24.265434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:24.266261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:24.266261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:24.266261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:24.266261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:24.266261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:24.266261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:24.266261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:24.266261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:24.266261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:24.266261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:24.266261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:24.266261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:24.266261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 13:54:24.266261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:24.266261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 436/884] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:54:24 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:24.274606 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05d020-854c-4d5d-9011-50f5cf1befd9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:24.417723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a05d020-854c-4d5d-9011-50f5cf1befd9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:24.456196 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af69ae2-1ddd-4a18-bf70-c90926f91180 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:24.457343 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af69ae2-1ddd-4a18-bf70-c90926f91180 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:24.457833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af69ae2-1ddd-4a18-bf70-c90926f91180 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:24.458333 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af69ae2-1ddd-4a18-bf70-c90926f91180 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:24.467920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af69ae2-1ddd-4a18-bf70-c90926f91180 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:24.468610 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af69ae2-1ddd-4a18-bf70-c90926f91180 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:24.470253 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7af69ae2-1ddd-4a18-bf70-c90926f91180 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:24.476537 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 449/885] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:54:24 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:24.483754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37db91-ee71-4a34-9101-99656bca2b17 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:24.596780 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad37db91-ee71-4a34-9101-99656bca2b17 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:24.636156 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:24.654669 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d6639-7341-4efe-8907-70294572b3c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:24.655378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d6639-7341-4efe-8907-70294572b3c3 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:24.655667 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d6639-7341-4efe-8907-70294572b3c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:24.655950 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5d6639-7341-4efe-8907-70294572b3c3 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:24.656703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d6639-7341-4efe-8907-70294572b3c3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:24.657020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d6639-7341-4efe-8907-70294572b3c3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:24.658500 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5d6639-7341-4efe-8907-70294572b3c3 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:24.664995 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 437/886] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:54:24 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:24.670615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af69ae2-1ddd-4a18-bf70-c90926f91180 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:24.717686 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:24.801544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af69ae2-1ddd-4a18-bf70-c90926f91180 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:24.843654 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e097af-22d8-4f0c-b46b-b1d83865f49d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:24.844733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e097af-22d8-4f0c-b46b-b1d83865f49d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:24.845232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e097af-22d8-4f0c-b46b-b1d83865f49d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:24.845712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e097af-22d8-4f0c-b46b-b1d83865f49d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:24.846846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e097af-22d8-4f0c-b46b-b1d83865f49d None admin] RBAC: Adding query filter params (interface=public, service_id=718bb0b718a1481e8464431d685a45c0, region_id=RegionOne) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:24.847403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e097af-22d8-4f0c-b46b-b1d83865f49d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:24.848980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e097af-22d8-4f0c-b46b-b1d83865f49d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:24.855501 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 450/887] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Jul 11 13:54:24 2022] GET /identity/v3/endpoints?service_id=718bb0b718a1481e8464431d685a45c0&interface=public®ion_id=RegionOne => generated 194 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:24.860969 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d6639-7341-4efe-8907-70294572b3c3 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:24.989412 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5d6639-7341-4efe-8907-70294572b3c3 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:25.042804 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d623f2-5344-4d90-b571-72ab1ba6711d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:25.044812 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d623f2-5344-4d90-b571-72ab1ba6711d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:25.045651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d623f2-5344-4d90-b571-72ab1ba6711d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:25.046474 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d623f2-5344-4d90-b571-72ab1ba6711d None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:25.048176 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d623f2-5344-4d90-b571-72ab1ba6711d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:25.050938 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d623f2-5344-4d90-b571-72ab1ba6711d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:25.068536 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 438/888] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Jul 11 13:54:24 2022] GET /identity/v3/services => generated 1268 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 13:54:25.312974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596179e-22b0-46e5-a4da-d217cbfba115 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:25.313675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596179e-22b0-46e5-a4da-d217cbfba115 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:25.314177 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3596179e-22b0-46e5-a4da-d217cbfba115 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:25.316149 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 451/889] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:25.323183 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6066735-5544-46a6-9562-575d4be379a9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:25.323853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6066735-5544-46a6-9562-575d4be379a9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:25.324492 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6066735-5544-46a6-9562-575d4be379a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:25.352285 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c6066735-5544-46a6-9562-575d4be379a9 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:25.354908 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c6066735-5544-46a6-9562-575d4be379a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:25.439143 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:25.483365 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 439/890] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:25.524678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596179e-22b0-46e5-a4da-d217cbfba115 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:25.625483 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b572703-51ec-4d55-89b0-8f50cf26fc7f None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:25.626423 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b572703-51ec-4d55-89b0-8f50cf26fc7f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:25.627410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b572703-51ec-4d55-89b0-8f50cf26fc7f None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:25.630059 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 440/891] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:25 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:54:25.636790 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb63a2-b968-409f-91e9-4d36cda3dd1e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:25.637293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb63a2-b968-409f-91e9-4d36cda3dd1e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:25.637584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6eb63a2-b968-409f-91e9-4d36cda3dd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:25.657657 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e6eb63a2-b968-409f-91e9-4d36cda3dd1e None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:25.658905 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e6eb63a2-b968-409f-91e9-4d36cda3dd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:25.676888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:25.683693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3596179e-22b0-46e5-a4da-d217cbfba115 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:25.731853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf9f82-3ca5-4716-8413-510bb959b1f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:25.733096 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf9f82-3ca5-4716-8413-510bb959b1f7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:25.733523 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf9f82-3ca5-4716-8413-510bb959b1f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:25.734016 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eaf9f82-3ca5-4716-8413-510bb959b1f7 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:25.735088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf9f82-3ca5-4716-8413-510bb959b1f7 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:25.736933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eaf9f82-3ca5-4716-8413-510bb959b1f7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:25.752506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-6eaf9f82-3ca5-4716-8413-510bb959b1f7 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 13:54:25.752506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:25.752506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:25.752506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:25.752506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:25.752506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:25.752506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:25.752506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:25.752506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 13:54:25.752506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:25.752506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:25.752506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:25.752506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:25.752506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:25.752506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:25.752506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:25.753549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:25.753549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:25.753549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:25.753549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:25.753549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:25.753549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:25.753549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:25.753549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:25.753549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:25.753549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:25.753549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:25.753549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:25.753549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:25.753549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:25.753549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:25.753549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:25.754470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:25.754470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:25.754470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:25.754470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:25.754470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:25.754470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:25.754470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:25.754470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:25.754470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:25.754470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:25.754470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:25.754470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:25.754470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:25.754470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 13:54:25.754470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:25.762366 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 452/892] 10.222.0.49 () {56 vars in 1132 bytes} [Mon Jul 11 13:54:25 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:25.762366 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf9f82-3ca5-4716-8413-510bb959b1f7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:25.764109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 441/893] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:25 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:25.789425 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb63a2-b968-409f-91e9-4d36cda3dd1e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:25.900915 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eaf9f82-3ca5-4716-8413-510bb959b1f7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:25.924990 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6eb63a2-b968-409f-91e9-4d36cda3dd1e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:25.938114 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:25.949617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c83bea9-5b6b-46c8-9b5a-ce67c210f3d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:25.950323 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83bea9-5b6b-46c8-9b5a-ce67c210f3d7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:25.950612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83bea9-5b6b-46c8-9b5a-ce67c210f3d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:25.950939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c83bea9-5b6b-46c8-9b5a-ce67c210f3d7 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:25.951668 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83bea9-5b6b-46c8-9b5a-ce67c210f3d7 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:25.951983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83bea9-5b6b-46c8-9b5a-ce67c210f3d7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:25.953364 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c83bea9-5b6b-46c8-9b5a-ce67c210f3d7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:25.958335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 453/894] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 11 13:54:25 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:25.959731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94587110-83a4-4d76-90bf-a519dca2382a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:25.960036 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94587110-83a4-4d76-90bf-a519dca2382a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:25.960311 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94587110-83a4-4d76-90bf-a519dca2382a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:25.962338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 454/895] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:25.965064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94587110-83a4-4d76-90bf-a519dca2382a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:25.984639 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c381ad-9d59-4ebb-aafd-76772279c3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:25.985502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c381ad-9d59-4ebb-aafd-76772279c3b0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:25.985785 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c381ad-9d59-4ebb-aafd-76772279c3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:25.986070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c381ad-9d59-4ebb-aafd-76772279c3b0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:26.003413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:26.003757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c381ad-9d59-4ebb-aafd-76772279c3b0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:54:26.017147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-02c381ad-9d59-4ebb-aafd-76772279c3b0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 13:54:26.017147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:26.017147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:26.017147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:26.017147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:26.017147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:26.017147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:26.017147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:26.017147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:26.017147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:26.017147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:26.017147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:26.017147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:54:26.017147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:54:26.017147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:54:26.017147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:54:26.026447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:54:26.026447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:54:26.026447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:54:26.026447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:54:26.026447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:26.026447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:26.026447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:26.026447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:26.026447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:26.026447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:26.026447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:26.026447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:26.026447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:26.026447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:26.026447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:26.026447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:26.028068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:26.028068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:26.028068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:26.028068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:26.028068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:54:26.028068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:54:26.028068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:54:26.028068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:54:26.028068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:54:26.028068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:54:26.028068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 13:54:26.028068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:26.028068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 442/896] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Jul 11 13:54:25 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:26.028068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc015eca-ffa7-4582-bb73-b194297acbeb None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:26.029495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc015eca-ffa7-4582-bb73-b194297acbeb None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:26.029495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc015eca-ffa7-4582-bb73-b194297acbeb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:26.049148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-fc015eca-ffa7-4582-bb73-b194297acbeb None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:26.049148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-fc015eca-ffa7-4582-bb73-b194297acbeb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:26.119061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94587110-83a4-4d76-90bf-a519dca2382a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:26.150522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 443/897] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:26 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:26.152355 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc015eca-ffa7-4582-bb73-b194297acbeb None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:26.163061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b5e1e-1d6d-4066-b0b7-e21f82910cb0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:26.164088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b5e1e-1d6d-4066-b0b7-e21f82910cb0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:26.165060 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b5e1e-1d6d-4066-b0b7-e21f82910cb0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:26.166281 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d60b5e1e-1d6d-4066-b0b7-e21f82910cb0 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:26.167519 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b5e1e-1d6d-4066-b0b7-e21f82910cb0 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:26.168430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b5e1e-1d6d-4066-b0b7-e21f82910cb0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:26.170976 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d60b5e1e-1d6d-4066-b0b7-e21f82910cb0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:26.177707 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 455/898] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 11 13:54:25 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:26.178695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b5e1e-1d6d-4066-b0b7-e21f82910cb0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:26.292981 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:26.299564 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc015eca-ffa7-4582-bb73-b194297acbeb None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:26.317367 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:26.330577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d60b5e1e-1d6d-4066-b0b7-e21f82910cb0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:26.351498 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cf5a5-756b-4d81-9b03-3e979af6cb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:26.352190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cf5a5-756b-4d81-9b03-3e979af6cb0b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:26.352471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cf5a5-756b-4d81-9b03-3e979af6cb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:26.352746 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35cf5a5-756b-4d81-9b03-3e979af6cb0b None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:26.362351 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cf5a5-756b-4d81-9b03-3e979af6cb0b None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:26.362737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cf5a5-756b-4d81-9b03-3e979af6cb0b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:26.364174 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35cf5a5-756b-4d81-9b03-3e979af6cb0b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:26.372447 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b2a16-06bd-4509-aab6-05a7f457cf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:26.374262 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b2a16-06bd-4509-aab6-05a7f457cf63 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:26.374599 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b2a16-06bd-4509-aab6-05a7f457cf63 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:26.374879 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b2a16-06bd-4509-aab6-05a7f457cf63 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:26.375628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b2a16-06bd-4509-aab6-05a7f457cf63 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:26.377011 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b2a16-06bd-4509-aab6-05a7f457cf63 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:26.377353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 444/899] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Jul 11 13:54:26 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:26.378366 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cf5a5-756b-4d81-9b03-3e979af6cb0b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:26.398531 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 456/900] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 11 13:54:26 2022] POST /identity/v3/services => generated 241 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:54:26.399301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b2a16-06bd-4509-aab6-05a7f457cf63 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:26.596524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35cf5a5-756b-4d81-9b03-3e979af6cb0b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:26.625859 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b2a16-06bd-4509-aab6-05a7f457cf63 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:26.675921 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce23882-b4dd-447d-b296-7cf733087253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:26.677857 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce23882-b4dd-447d-b296-7cf733087253 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:26.678649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce23882-b4dd-447d-b296-7cf733087253 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:26.679350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce23882-b4dd-447d-b296-7cf733087253 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:26.679728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776ec93-ff62-45f4-b343-eb8fb1deedec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:26.680806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776ec93-ff62-45f4-b343-eb8fb1deedec None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:26.681091 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776ec93-ff62-45f4-b343-eb8fb1deedec None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:26.681357 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0776ec93-ff62-45f4-b343-eb8fb1deedec None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:26.682576 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce23882-b4dd-447d-b296-7cf733087253 None admin] RBAC: Adding query filter params (interface=public, service_id=76378b3bd1004d2cbf78df00e67408fe, region_id=RegionOne) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:26.683168 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce23882-b4dd-447d-b296-7cf733087253 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:26.687126 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce23882-b4dd-447d-b296-7cf733087253 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:26.698152 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 445/901] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Jul 11 13:54:26 2022] GET /identity/v3/endpoints?service_id=76378b3bd1004d2cbf78df00e67408fe&interface=public®ion_id=RegionOne => generated 194 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:26.701450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776ec93-ff62-45f4-b343-eb8fb1deedec None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:26.703823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce23882-b4dd-447d-b296-7cf733087253 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:26.705112 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0776ec93-ff62-45f4-b343-eb8fb1deedec None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:26.726219 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:26.738779 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:26.748636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-0776ec93-ff62-45f4-b343-eb8fb1deedec None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:26.748636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:26.748636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:26.748636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:26.748636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:26.748636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:26.748636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:26.748636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:26.748636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:26.748636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:26.748636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:26.748636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:26.748636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:26.748636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:26.748636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:26.749959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:26.749959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:26.749959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:26.749959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:26.749959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:26.749959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:26.749959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:26.749959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:26.749959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:26.749959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:26.749959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:26.749959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:26.749959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:26.749959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:26.749959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:26.749959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:26.749959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:26.751083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:26.751083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:26.751083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:26.751083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:26.751083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 457/902] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:26.756235 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776ec93-ff62-45f4-b343-eb8fb1deedec None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:26.851349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce23882-b4dd-447d-b296-7cf733087253 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:26.871135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0776ec93-ff62-45f4-b343-eb8fb1deedec None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:26.891324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a204cd7-e97e-447f-ab0b-5f49db3f744e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:26.892234 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a204cd7-e97e-447f-ab0b-5f49db3f744e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:26.892396 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a204cd7-e97e-447f-ab0b-5f49db3f744e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:26.892800 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a204cd7-e97e-447f-ab0b-5f49db3f744e None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:26.893513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a204cd7-e97e-447f-ab0b-5f49db3f744e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:26.894958 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a204cd7-e97e-447f-ab0b-5f49db3f744e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:26.908764 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 446/903] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Jul 11 13:54:26 2022] GET /identity/v3/services => generated 1497 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 13:54:26.914264 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ea9be-f45e-4bbe-962c-1183373a5a9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:26.915624 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ea9be-f45e-4bbe-962c-1183373a5a9f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:26.916148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ea9be-f45e-4bbe-962c-1183373a5a9f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:26.916609 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f46ea9be-f45e-4bbe-962c-1183373a5a9f None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:26.917643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ea9be-f45e-4bbe-962c-1183373a5a9f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:26.918166 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ea9be-f45e-4bbe-962c-1183373a5a9f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:26.920259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f46ea9be-f45e-4bbe-962c-1183373a5a9f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:26.946047 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 458/904] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:26.948364 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a204cd7-e97e-447f-ab0b-5f49db3f744e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:27.132893 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:27.157964 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a204cd7-e97e-447f-ab0b-5f49db3f744e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:27.223165 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ca18757-462f-4909-b443-1bfb845da37b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:27.224589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca18757-462f-4909-b443-1bfb845da37b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:27.225093 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca18757-462f-4909-b443-1bfb845da37b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:27.225577 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ca18757-462f-4909-b443-1bfb845da37b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:27.240702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca18757-462f-4909-b443-1bfb845da37b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:27.243483 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ca18757-462f-4909-b443-1bfb845da37b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:27.272745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-9ca18757-462f-4909-b443-1bfb845da37b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 13:54:27.272745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:27.272745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:27.272745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:27.272745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:27.272745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:27.272745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:27.272745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:27.272745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:27.272745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:27.272745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:27.272745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:27.272745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:54:27.272745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:54:27.272745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:54:27.272745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:54:27.273742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:27.273742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:27.273742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:54:27.273742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:27.273742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:54:27.273742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:27.273742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:27.273742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:27.273742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:27.273742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:27.273742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:27.273742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:27.273742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:27.273742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:27.273742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:27.273742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:27.273742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:27.273742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:27.274819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:27.274819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:27.274819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:54:27.274819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:54:27.274819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:54:27.274819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:54:27.274819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:54:27.274819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:54:27.274819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 13:54:27.274819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:27.281996 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 447/905] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 11 13:54:26 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 327 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:27.284170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ea9be-f45e-4bbe-962c-1183373a5a9f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:27.302021 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f32e39-048f-4b8f-a192-6ba389e6665b None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:27.302537 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f32e39-048f-4b8f-a192-6ba389e6665b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:27.303008 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f32e39-048f-4b8f-a192-6ba389e6665b None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:27.307059 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 448/906] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:27 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:27.313152 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0f11c-0f4e-4f24-97a1-e0230d95185a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:27.313681 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0f11c-0f4e-4f24-97a1-e0230d95185a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:27.314135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0f11c-0f4e-4f24-97a1-e0230d95185a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:27.333429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1ee0f11c-0f4e-4f24-97a1-e0230d95185a None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:27.335157 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1ee0f11c-0f4e-4f24-97a1-e0230d95185a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:27.412217 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:27.448420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 449/907] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:27 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:54:27.470805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0f11c-0f4e-4f24-97a1-e0230d95185a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:27.488837 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f46ea9be-f45e-4bbe-962c-1183373a5a9f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:27.536504 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b2633-d67f-4223-a7c1-85c395c1888f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:27.537541 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b2633-d67f-4223-a7c1-85c395c1888f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:27.538038 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b2633-d67f-4223-a7c1-85c395c1888f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:27.538561 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b2633-d67f-4223-a7c1-85c395c1888f None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:27.539976 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b2633-d67f-4223-a7c1-85c395c1888f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:27.540496 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b2633-d67f-4223-a7c1-85c395c1888f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:27.543346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-081b2633-d67f-4223-a7c1-85c395c1888f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:27.568642 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 459/908] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Jul 11 13:54:27 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:27.578478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b2633-d67f-4223-a7c1-85c395c1888f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:27.643038 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:27.685598 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ee0f11c-0f4e-4f24-97a1-e0230d95185a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:27.734090 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f4237-70db-4f2e-b224-353157555e92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:27.735773 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f4237-70db-4f2e-b224-353157555e92 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:27.736336 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f4237-70db-4f2e-b224-353157555e92 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:27.737446 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6f4237-70db-4f2e-b224-353157555e92 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:27.738510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f4237-70db-4f2e-b224-353157555e92 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:27.740815 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6f4237-70db-4f2e-b224-353157555e92 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:27.754918 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b2633-d67f-4223-a7c1-85c395c1888f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:27.757271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-0a6f4237-70db-4f2e-b224-353157555e92 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 13:54:27.757271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:27.757271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:27.757271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:27.757271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:27.757271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:27.757271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:27.757271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:27.757271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 13:54:27.757271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:27.757271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:27.757271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:27.757271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:27.757271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:27.757271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:27.757271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:27.757271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:27.758283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:27.758283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:27.758283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:27.758283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:27.758283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:27.758283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:27.758283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:27.758283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:27.758283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:27.758283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:27.758283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:27.758283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:27.758283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:27.758283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:27.758283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:27.758283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:27.759324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:27.759324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:27.759324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:27.759324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:27.759324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:27.759324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:27.759324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:27.759324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:27.759324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:27.759324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:27.759324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:27.759324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:27.759324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 11 13:54:27.759324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:27.762401 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 450/909] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:54:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:27.767848 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f4237-70db-4f2e-b224-353157555e92 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:27.801537 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:27.840159 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41720a07-7647-4874-aac7-f643cb8c13a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:27.841249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41720a07-7647-4874-aac7-f643cb8c13a6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:27.842340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41720a07-7647-4874-aac7-f643cb8c13a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:27.842962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41720a07-7647-4874-aac7-f643cb8c13a6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:27.895025 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6f4237-70db-4f2e-b224-353157555e92 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:27.936660 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f48a51-6afa-47d5-bb7d-53715c21c638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:27.937739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f48a51-6afa-47d5-bb7d-53715c21c638 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:27.938260 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f48a51-6afa-47d5-bb7d-53715c21c638 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:27.938833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f48a51-6afa-47d5-bb7d-53715c21c638 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:27.939866 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f48a51-6afa-47d5-bb7d-53715c21c638 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:27.940397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f48a51-6afa-47d5-bb7d-53715c21c638 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:27.942046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f48a51-6afa-47d5-bb7d-53715c21c638 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:27.952848 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 451/910] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:54:27 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:27.954156 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f48a51-6afa-47d5-bb7d-53715c21c638 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:27.965430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:27.992039 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41720a07-7647-4874-aac7-f643cb8c13a6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:27.995569 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41720a07-7647-4874-aac7-f643cb8c13a6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:28.073796 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f48a51-6afa-47d5-bb7d-53715c21c638 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:28.114173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8c30b-591d-4347-9828-dedb1586cce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:28.115050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8c30b-591d-4347-9828-dedb1586cce3 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:28.115302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8c30b-591d-4347-9828-dedb1586cce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:28.115546 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb8c30b-591d-4347-9828-dedb1586cce3 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:28.116273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8c30b-591d-4347-9828-dedb1586cce3 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:28.116589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8c30b-591d-4347-9828-dedb1586cce3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:28.118187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb8c30b-591d-4347-9828-dedb1586cce3 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:28.129671 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 452/911] 10.222.0.49 () {56 vars in 1114 bytes} [Mon Jul 11 13:54:27 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:28.130721 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8c30b-591d-4347-9828-dedb1586cce3 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:28.215813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:28.220830 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 460/912] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:54:27 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 643 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 13:54:28.231869 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41720a07-7647-4874-aac7-f643cb8c13a6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:28.253796 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb8c30b-591d-4347-9828-dedb1586cce3 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:28.269229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:28.293807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df16f1-8c57-4784-97e7-e3ba4b8a49f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:28.294283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df16f1-8c57-4784-97e7-e3ba4b8a49f8 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:28.295119 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df16f1-8c57-4784-97e7-e3ba4b8a49f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:28.295412 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4df16f1-8c57-4784-97e7-e3ba4b8a49f8 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:28.296203 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df16f1-8c57-4784-97e7-e3ba4b8a49f8 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:28.297834 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4df16f1-8c57-4784-97e7-e3ba4b8a49f8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:28.322966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 453/913] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Jul 11 13:54:28 2022] POST /identity/v3/endpoints => generated 346 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:54:28.454204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41720a07-7647-4874-aac7-f643cb8c13a6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:28.539856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf9d8b-3cec-4aaa-842a-639f6d989bae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:28.541638 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf9d8b-3cec-4aaa-842a-639f6d989bae None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:28.542431 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf9d8b-3cec-4aaa-842a-639f6d989bae None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:28.543176 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cdf9d8b-3cec-4aaa-842a-639f6d989bae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:28.558961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf9d8b-3cec-4aaa-842a-639f6d989bae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:28.560506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cdf9d8b-3cec-4aaa-842a-639f6d989bae None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:28.580955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-3cdf9d8b-3cec-4aaa-842a-639f6d989bae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:28.580955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:28.580955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:28.580955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:28.580955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:28.580955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:28.580955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:28.580955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:28.580955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:28.580955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:28.580955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:28.580955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:28.580955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:28.580955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:28.580955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:28.582961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:28.582961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:28.582961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:28.582961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:28.582961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:28.582961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:28.582961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:28.582961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:28.582961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:28.582961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:28.582961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:28.582961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:28.582961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:28.582961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:28.582961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:28.582961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:28.582961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:28.584731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:28.584731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:28.584731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:28.584731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:28.584731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 461/914] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:28.595311 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df16f1-8c57-4784-97e7-e3ba4b8a49f8 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:28.754565 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4df16f1-8c57-4784-97e7-e3ba4b8a49f8 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:28.792638 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:28.798145 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40a46a-c90d-4f92-9e0f-bf5a57893839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:28.799879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40a46a-c90d-4f92-9e0f-bf5a57893839 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:28.800709 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40a46a-c90d-4f92-9e0f-bf5a57893839 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:28.801467 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d40a46a-c90d-4f92-9e0f-bf5a57893839 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:28.803230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40a46a-c90d-4f92-9e0f-bf5a57893839 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:28.804075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40a46a-c90d-4f92-9e0f-bf5a57893839 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:28.807381 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d40a46a-c90d-4f92-9e0f-bf5a57893839 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:28.838466 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 454/915] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 240 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:28.842456 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf9d8b-3cec-4aaa-842a-639f6d989bae None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:28.916772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:28.958052 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cdf9d8b-3cec-4aaa-842a-639f6d989bae None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:28.995211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-403c4e4d-cde7-4e2a-9595-4b0d42788b8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:28.996023 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c4e4d-cde7-4e2a-9595-4b0d42788b8c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:28.996299 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c4e4d-cde7-4e2a-9595-4b0d42788b8c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:28.996581 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-403c4e4d-cde7-4e2a-9595-4b0d42788b8c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:29.004463 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c4e4d-cde7-4e2a-9595-4b0d42788b8c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:29.005929 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-403c4e4d-cde7-4e2a-9595-4b0d42788b8c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:29.021429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-403c4e4d-cde7-4e2a-9595-4b0d42788b8c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:29.021429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:29.021429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:29.021429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:29.021429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:29.021429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:29.021429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:29.021429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:29.021429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:29.021429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:29.021429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:29.021429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:29.021429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:54:29.021429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:54:29.021429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:54:29.023967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:54:29.023967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:29.023967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:29.023967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:29.023967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:29.023967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:29.023967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:29.023967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:29.023967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:29.023967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:29.023967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:29.023967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:29.023967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:29.023967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:29.023967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:29.023967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:29.023967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:29.025997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:54:29.025997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:54:29.025997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:54:29.025997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:54:29.025997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:54:29.025997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:54:29.025997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:29.025997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:29.025997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 462/916] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:29.037383 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40a46a-c90d-4f92-9e0f-bf5a57893839 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:29.162612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd556d5-1c94-4f16-b907-aacd45f2cd17 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:29.162870 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd556d5-1c94-4f16-b907-aacd45f2cd17 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:29.163191 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd556d5-1c94-4f16-b907-aacd45f2cd17 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:29.164785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 463/917] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:54:29.170444 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ce38a-3c06-4ec0-a0bd-459bb5381087 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:29.170715 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ce38a-3c06-4ec0-a0bd-459bb5381087 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:29.171223 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-995ce38a-3c06-4ec0-a0bd-459bb5381087 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:29.189924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-995ce38a-3c06-4ec0-a0bd-459bb5381087 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:29.191270 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-995ce38a-3c06-4ec0-a0bd-459bb5381087 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:29.235905 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d40a46a-c90d-4f92-9e0f-bf5a57893839 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:29.270661 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 464/918] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:54:29.289035 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ce38a-3c06-4ec0-a0bd-459bb5381087 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:29.305428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ecfc1-069e-47bd-abfe-47344b5697c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:29.307014 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ecfc1-069e-47bd-abfe-47344b5697c2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:29.307881 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ecfc1-069e-47bd-abfe-47344b5697c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:29.308614 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9ecfc1-069e-47bd-abfe-47344b5697c2 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:29.310874 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ecfc1-069e-47bd-abfe-47344b5697c2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:29.311651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ecfc1-069e-47bd-abfe-47344b5697c2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:29.314608 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9ecfc1-069e-47bd-abfe-47344b5697c2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:29.345660 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 455/919] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:54:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:29.349603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ecfc1-069e-47bd-abfe-47344b5697c2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:29.350429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:29.398771 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-995ce38a-3c06-4ec0-a0bd-459bb5381087 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:29.435936 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237e485-e345-4b0b-adf9-7c60921a3a7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:29.436555 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237e485-e345-4b0b-adf9-7c60921a3a7e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:29.436845 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237e485-e345-4b0b-adf9-7c60921a3a7e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:29.437126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1237e485-e345-4b0b-adf9-7c60921a3a7e None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:29.437795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1237e485-e345-4b0b-adf9-7c60921a3a7e None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:29.439215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1237e485-e345-4b0b-adf9-7c60921a3a7e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:29.452385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-1237e485-e345-4b0b-adf9-7c60921a3a7e None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 13:54:29.452385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:29.452385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:29.452385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:29.452385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:29.452385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:29.452385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:29.452385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:29.452385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 13:54:29.452385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:29.452385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:29.452385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:29.452385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:29.452385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:29.452385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:29.452385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:29.454087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:29.454087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:29.454087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:29.454087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:29.454087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:29.454087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:29.454087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:29.454087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:29.454087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:29.454087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:29.454087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:29.454087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:29.454087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:29.454087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:29.454087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:29.454087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:29.456254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:29.456254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:29.456254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:29.456254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:29.456254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:29.456254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:29.456254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:29.456254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:29.456254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:29.456254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:29.456254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:29.456254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:29.456254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:29.456254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 11 13:54:29.456254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:29.458640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 465/920] 10.222.0.49 () {56 vars in 1132 bytes} [Mon Jul 11 13:54:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:29.459675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237e485-e345-4b0b-adf9-7c60921a3a7e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:29.462747 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:29.526368 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9ecfc1-069e-47bd-abfe-47344b5697c2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:29.569525 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1237e485-e345-4b0b-adf9-7c60921a3a7e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:29.574718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172cbb6-c714-4d68-8833-6f92deae2fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:29.575776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172cbb6-c714-4d68-8833-6f92deae2fd1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:29.576250 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172cbb6-c714-4d68-8833-6f92deae2fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:29.576710 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5172cbb6-c714-4d68-8833-6f92deae2fd1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:29.578276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172cbb6-c714-4d68-8833-6f92deae2fd1 None admin] RBAC: Adding query filter params (role.id=836753c7d9b44fde817a38c1566d0659, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=52076f59f68f4d77a0dadbf7a98d2b57) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:29.578821 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172cbb6-c714-4d68-8833-6f92deae2fd1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:29.581010 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5172cbb6-c714-4d68-8833-6f92deae2fd1 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:29.588852 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:29.598625 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 456/921] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:54:29 2022] GET /identity/v3/role_assignments?user.id=52076f59f68f4d77a0dadbf7a98d2b57&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=836753c7d9b44fde817a38c1566d0659 => generated 259 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:29.614587 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dc0d939-01fb-4466-bfd3-29b2d5cd5362 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:29.615332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0d939-01fb-4466-bfd3-29b2d5cd5362 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:29.615624 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0d939-01fb-4466-bfd3-29b2d5cd5362 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:29.615920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc0d939-01fb-4466-bfd3-29b2d5cd5362 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:29.616651 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0d939-01fb-4466-bfd3-29b2d5cd5362 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:29.616964 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0d939-01fb-4466-bfd3-29b2d5cd5362 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:29.618421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dc0d939-01fb-4466-bfd3-29b2d5cd5362 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:29.625265 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 466/922] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 11 13:54:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:29.631479 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172cbb6-c714-4d68-8833-6f92deae2fd1 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:29.670534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37392534-af52-4863-8508-aecc16273e47 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:29.670876 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37392534-af52-4863-8508-aecc16273e47 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:29.671092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37392534-af52-4863-8508-aecc16273e47 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:29.672473 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 467/923] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:54:29.677923 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df489e-c416-4ddb-b6d8-fe24d5aef7c2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:29.678205 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df489e-c416-4ddb-b6d8-fe24d5aef7c2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:29.678503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65df489e-c416-4ddb-b6d8-fe24d5aef7c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:29.699324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-65df489e-c416-4ddb-b6d8-fe24d5aef7c2 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:29.700550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-65df489e-c416-4ddb-b6d8-fe24d5aef7c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:29.767075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5172cbb6-c714-4d68-8833-6f92deae2fd1 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:29.783524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:29.808375 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 468/924] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:29 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:54:29.822344 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a66e67-9efd-469c-af95-272287144ad6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:29.823093 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a66e67-9efd-469c-af95-272287144ad6 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:29.823375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a66e67-9efd-469c-af95-272287144ad6 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:29.823697 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a66e67-9efd-469c-af95-272287144ad6 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:29.824417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a66e67-9efd-469c-af95-272287144ad6 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:29.824723 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a66e67-9efd-469c-af95-272287144ad6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:29.826242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a66e67-9efd-469c-af95-272287144ad6 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:29.832321 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 457/925] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 11 13:54:29 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:29.839721 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df489e-c416-4ddb-b6d8-fe24d5aef7c2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:29.842390 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a66e67-9efd-469c-af95-272287144ad6 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:29.965638 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65df489e-c416-4ddb-b6d8-fe24d5aef7c2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:29.971743 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a66e67-9efd-469c-af95-272287144ad6 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:29.985419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:30.011518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1e697-f8d1-45ae-bb3b-9eb4e3259a15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:30.012490 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1e697-f8d1-45ae-bb3b-9eb4e3259a15 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:30.012911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1e697-f8d1-45ae-bb3b-9eb4e3259a15 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:30.013453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7f1e697-f8d1-45ae-bb3b-9eb4e3259a15 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:30.014399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1e697-f8d1-45ae-bb3b-9eb4e3259a15 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:30.015189 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98010238-0b93-4244-9829-bfc5b980da13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:30.015940 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7f1e697-f8d1-45ae-bb3b-9eb4e3259a15 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:30.016198 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98010238-0b93-4244-9829-bfc5b980da13 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:30.016633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98010238-0b93-4244-9829-bfc5b980da13 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:30.017061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98010238-0b93-4244-9829-bfc5b980da13 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:30.017973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98010238-0b93-4244-9829-bfc5b980da13 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:30.021227 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98010238-0b93-4244-9829-bfc5b980da13 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:30.031757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-c7f1e697-f8d1-45ae-bb3b-9eb4e3259a15 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 13:54:30.031757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:30.031757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:30.031757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:30.031757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:30.031757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:30.031757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:30.031757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:30.031757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 13:54:30.031757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:30.031757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:30.031757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:30.031757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:30.031757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:30.031757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:30.031757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:30.031757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:30.033065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:30.033065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:30.033065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:30.033065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:30.033065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:30.033065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:30.033065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:30.033065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:30.033065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:30.033065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:30.033065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:30.033065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:30.033065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:30.033065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:30.033065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:30.033065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:30.034130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:30.034130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:30.034130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:30.034130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:30.034130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:30.034130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:30.034130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:30.034130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:30.034130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:30.034130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:30.034130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:30.034130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:30.034130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 13:54:30.034130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:30.038482 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 458/926] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 11 13:54:29 2022] GET /identity/v3/services/object-store => generated 93 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:30.042605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1e697-f8d1-45ae-bb3b-9eb4e3259a15 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:30.062397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 469/927] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Jul 11 13:54:29 2022] POST /identity/v3/endpoints => generated 347 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:54:30.121470 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:30.240034 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7f1e697-f8d1-45ae-bb3b-9eb4e3259a15 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:30.319795 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb969951-bf8d-44f9-8da2-d7c5dba2995e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:30.321514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb969951-bf8d-44f9-8da2-d7c5dba2995e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:30.322341 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb969951-bf8d-44f9-8da2-d7c5dba2995e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:30.323132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb969951-bf8d-44f9-8da2-d7c5dba2995e None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:30.324802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb969951-bf8d-44f9-8da2-d7c5dba2995e None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:30.326174 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb969951-bf8d-44f9-8da2-d7c5dba2995e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:30.329211 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb969951-bf8d-44f9-8da2-d7c5dba2995e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:30.339621 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 459/928] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 11 13:54:30 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 297 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:30.347533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98010238-0b93-4244-9829-bfc5b980da13 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:30.479421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98010238-0b93-4244-9829-bfc5b980da13 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:30.516425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd31b10e-759f-40da-8b5e-63197796f754 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:30.517472 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31b10e-759f-40da-8b5e-63197796f754 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:30.517963 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31b10e-759f-40da-8b5e-63197796f754 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:30.518598 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd31b10e-759f-40da-8b5e-63197796f754 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:30.519711 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31b10e-759f-40da-8b5e-63197796f754 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:30.520217 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31b10e-759f-40da-8b5e-63197796f754 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:30.521783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd31b10e-759f-40da-8b5e-63197796f754 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:30.527709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 470/929] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 11 13:54:30 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:30.535435 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb969951-bf8d-44f9-8da2-d7c5dba2995e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:30.647306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:30.652001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb969951-bf8d-44f9-8da2-d7c5dba2995e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:30.689536 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d216edc-7a51-4be2-a8af-299cd0b13e3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:30.690208 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d216edc-7a51-4be2-a8af-299cd0b13e3e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:30.690524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d216edc-7a51-4be2-a8af-299cd0b13e3e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:30.690805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d216edc-7a51-4be2-a8af-299cd0b13e3e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:30.691612 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d216edc-7a51-4be2-a8af-299cd0b13e3e None admin] RBAC: Adding query filter params (interface=public, service_id=11e9e9396b5642a0867ced76b22701e5, region_id=RegionOne) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:30.691928 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d216edc-7a51-4be2-a8af-299cd0b13e3e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:30.693316 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d216edc-7a51-4be2-a8af-299cd0b13e3e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:30.699587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 460/930] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Jul 11 13:54:30 2022] GET /identity/v3/endpoints?service_id=11e9e9396b5642a0867ced76b22701e5&interface=public®ion_id=RegionOne => generated 194 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:30.705849 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ffc4b-9dca-4cf8-a0fa-37729a4be48d None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:30.706151 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ffc4b-9dca-4cf8-a0fa-37729a4be48d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:30.706469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e14ffc4b-9dca-4cf8-a0fa-37729a4be48d None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:30.707327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d216edc-7a51-4be2-a8af-299cd0b13e3e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:30.707975 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 471/931] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:30.713199 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1914b-5313-4590-8267-a4c2bb0ecf2f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:30.713501 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1914b-5313-4590-8267-a4c2bb0ecf2f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:30.713784 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05f1914b-5313-4590-8267-a4c2bb0ecf2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:30.730381 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-05f1914b-5313-4590-8267-a4c2bb0ecf2f None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:30.731579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-05f1914b-5313-4590-8267-a4c2bb0ecf2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:30.764107 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:30.814664 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 472/932] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:30 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:54:30.833617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1914b-5313-4590-8267-a4c2bb0ecf2f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:30.840499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d216edc-7a51-4be2-a8af-299cd0b13e3e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:30.878074 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea1cbb-5aa1-48a8-b1dd-401162f0e6ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:30.879239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea1cbb-5aa1-48a8-b1dd-401162f0e6ae None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:30.879729 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea1cbb-5aa1-48a8-b1dd-401162f0e6ae None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:30.880259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ea1cbb-5aa1-48a8-b1dd-401162f0e6ae None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:30.881200 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea1cbb-5aa1-48a8-b1dd-401162f0e6ae None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:30.882933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ea1cbb-5aa1-48a8-b1dd-401162f0e6ae None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:30.891208 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 461/933] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Jul 11 13:54:30 2022] GET /identity/v3/services => generated 1497 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 13:54:30.949995 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05f1914b-5313-4590-8267-a4c2bb0ecf2f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:30.990865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa672e8-88be-4741-9f59-ec644b4ec157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:30.992492 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa672e8-88be-4741-9f59-ec644b4ec157 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:30.992983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa672e8-88be-4741-9f59-ec644b4ec157 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:30.993461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa672e8-88be-4741-9f59-ec644b4ec157 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:30.994438 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa672e8-88be-4741-9f59-ec644b4ec157 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:30.996129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa672e8-88be-4741-9f59-ec644b4ec157 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:30.998325 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: WARNING keystone.server.flask.common [None req-bfa672e8-88be-4741-9f59-ec644b4ec157 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 11 13:54:31.020779 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.sql.core [None req-bfa672e8-88be-4741-9f59-ec644b4ec157 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 11 13:54:31.020779 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 11 13:54:31.020779 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [parameters: {'id': '17fa17691c8a49aa9af05c55ed659216', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 11 13:54:31.020779 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80248) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 11 13:54:31.029614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-bfa672e8-88be-4741-9f59-ec644b4ec157 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 17fa17691c8a49aa9af05c55ed659216.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 17fa17691c8a49aa9af05c55ed659216. Jul 11 13:54:31.029614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:31.029614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 13:54:31.029614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 13:54:31.029614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 13:54:31.029614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 13:54:31.029614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 13:54:31.029614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 13:54:31.029614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 13:54:31.029614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application conn.query(q) Jul 11 13:54:31.029614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 13:54:31.029614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 13:54:31.029614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 13:54:31.030560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application result.read() Jul 11 13:54:31.030560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 13:54:31.030560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 13:54:31.030560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 13:54:31.030560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 13:54:31.030560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 13:54:31.030560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 13:54:31.030560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 13:54:31.030560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 13:54:31.030560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 11 13:54:31.030560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:31.030560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 11 13:54:31.030560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:31.030560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:31.030560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 11 13:54:31.030560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 11 13:54:31.030560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 11 13:54:31.030560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 11 13:54:31.031533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 13:54:31.031533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application next(self.gen) Jul 11 13:54:31.031533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 11 13:54:31.031533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application yield resource Jul 11 13:54:31.031533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 13:54:31.031533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application next(self.gen) Jul 11 13:54:31.031533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 11 13:54:31.031533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.session.rollback() Jul 11 13:54:31.031533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 11 13:54:31.031533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.force_reraise() Jul 11 13:54:31.031533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 11 13:54:31.031533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise self.value Jul 11 13:54:31.031533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 11 13:54:31.031533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 11 13:54:31.031533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 11 13:54:31.031533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application session.commit() Jul 11 13:54:31.031533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 11 13:54:31.032413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 11 13:54:31.032413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 11 13:54:31.032413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._prepare_impl() Jul 11 13:54:31.032413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 11 13:54:31.032413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.session.flush() Jul 11 13:54:31.032413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 11 13:54:31.032413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._flush(objects) Jul 11 13:54:31.032413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 11 13:54:31.032413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 11 13:54:31.032413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 11 13:54:31.032413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application compat.raise_( Jul 11 13:54:31.032413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 13:54:31.032413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception Jul 11 13:54:31.032413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 11 13:54:31.032413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application flush_context.execute() Jul 11 13:54:31.032413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 11 13:54:31.032413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rec.execute(self) Jul 11 13:54:31.033379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 11 13:54:31.033379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 11 13:54:31.033379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 11 13:54:31.033379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 11 13:54:31.033379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 11 13:54:31.033379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 11 13:54:31.033379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 11 13:54:31.033379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 11 13:54:31.033379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 11 13:54:31.033379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 11 13:54:31.033379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 11 13:54:31.033379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 11 13:54:31.033379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 11 13:54:31.033379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 11 13:54:31.033379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 11 13:54:31.034175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 11 13:54:31.034175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 13:54:31.034175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception Jul 11 13:54:31.034175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 13:54:31.034175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 13:54:31.034175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 13:54:31.034175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 13:54:31.034175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 13:54:31.034175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 13:54:31.034175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 13:54:31.034175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application conn.query(q) Jul 11 13:54:31.034175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 13:54:31.034175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 13:54:31.034175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 13:54:31.034175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application result.read() Jul 11 13:54:31.034175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 13:54:31.034175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 13:54:31.036917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 13:54:31.036917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 13:54:31.036917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 13:54:31.036917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 13:54:31.036917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 13:54:31.036917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 13:54:31.036917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 11 13:54:31.036917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 11 13:54:31.036917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [parameters: {'id': '17fa17691c8a49aa9af05c55ed659216', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 11 13:54:31.036917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 11 13:54:31.036917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:31.036917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:31.036917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:31.036917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:31.036917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 11 13:54:31.038811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 11 13:54:31.038811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 11 13:54:31.038811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 11 13:54:31.038811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 11 13:54:31.038811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:31.038811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:31.038811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:31.038811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:31.038811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:31.038811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:31.038811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:31.038811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:31.038811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:31.038811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:31.038811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:31.038811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:31.038811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:31.040954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:31.040954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 11 13:54:31.040954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 11 13:54:31.040954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:31.040954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:31.040954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 11 13:54:31.040954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 11 13:54:31.040954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 17fa17691c8a49aa9af05c55ed659216. Jul 11 13:54:31.040954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:31.043071 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 473/934] 10.222.0.49 () {60 vars in 1140 bytes} [Mon Jul 11 13:54:30 2022] POST /identity/v3/projects => generated 262 bytes in 210 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:31.049718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea1cbb-5aa1-48a8-b1dd-401162f0e6ae None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:31.083529 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:31.176149 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ea1cbb-5aa1-48a8-b1dd-401162f0e6ae None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:31.212189 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54f1b71a-6e6e-42f6-9c7c-fc22068c2193 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:31.213031 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1b71a-6e6e-42f6-9c7c-fc22068c2193 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:31.213319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1b71a-6e6e-42f6-9c7c-fc22068c2193 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:31.213595 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54f1b71a-6e6e-42f6-9c7c-fc22068c2193 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:31.220972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f1b71a-6e6e-42f6-9c7c-fc22068c2193 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:31.222511 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54f1b71a-6e6e-42f6-9c7c-fc22068c2193 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:31.238363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-54f1b71a-6e6e-42f6-9c7c-fc22068c2193 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:31.238363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:31.238363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:31.238363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:31.238363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:31.238363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:31.238363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:31.238363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:31.238363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:31.238363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:31.238363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:31.238363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:31.238363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:54:31.238363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:54:31.238363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:54:31.239308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:54:31.239308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:31.239308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:31.239308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:31.239308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:31.239308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:31.239308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:31.239308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:31.239308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:31.239308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:31.239308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:31.239308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:31.239308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:31.239308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:31.239308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:31.239308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:31.239308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:31.240620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:54:31.240620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:54:31.240620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:54:31.240620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:54:31.240620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:54:31.240620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:54:31.240620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:31.240620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:31.240620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 462/935] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:31 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:31.248571 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa672e8-88be-4741-9f59-ec644b4ec157 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:31.272277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:31.368860 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa672e8-88be-4741-9f59-ec644b4ec157 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:31.407622 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a357ba55-dd46-4c73-b138-3ef0a6a8833d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:31.408736 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357ba55-dd46-4c73-b138-3ef0a6a8833d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:31.409223 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357ba55-dd46-4c73-b138-3ef0a6a8833d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:31.409694 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a357ba55-dd46-4c73-b138-3ef0a6a8833d None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:31.410697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a357ba55-dd46-4c73-b138-3ef0a6a8833d None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:31.411229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a357ba55-dd46-4c73-b138-3ef0a6a8833d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:31.413196 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a357ba55-dd46-4c73-b138-3ef0a6a8833d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:31.436873 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 474/936] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 11 13:54:31 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:31.769766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9151d-7729-45b5-a0b1-ae2bedb66f57 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:31.770119 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9151d-7729-45b5-a0b1-ae2bedb66f57 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:31.770377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e9151d-7729-45b5-a0b1-ae2bedb66f57 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:31.772078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 463/937] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:31.778102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1064b-0e29-4af4-be5d-e3bfcc31125f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:31.778491 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1064b-0e29-4af4-be5d-e3bfcc31125f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:31.778879 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b1064b-0e29-4af4-be5d-e3bfcc31125f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:31.798383 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-83b1064b-0e29-4af4-be5d-e3bfcc31125f None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:31.799960 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-83b1064b-0e29-4af4-be5d-e3bfcc31125f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:31.878424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 475/938] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:31 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:54:31.895596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9151d-7729-45b5-a0b1-ae2bedb66f57 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:31.986925 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:32.004304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e9151d-7729-45b5-a0b1-ae2bedb66f57 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:32.042262 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac7af8-82d2-421b-8ac9-ed67b08cd79d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:32.043026 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac7af8-82d2-421b-8ac9-ed67b08cd79d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:32.043312 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac7af8-82d2-421b-8ac9-ed67b08cd79d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:32.043595 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6ac7af8-82d2-421b-8ac9-ed67b08cd79d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:32.055834 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6ac7af8-82d2-421b-8ac9-ed67b08cd79d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:54:32.065002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-d6ac7af8-82d2-421b-8ac9-ed67b08cd79d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 13:54:32.065002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:32.065002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:32.065002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:32.065002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:32.065002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:32.065002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:32.065002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:32.065002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:32.065002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:32.065002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:32.065002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:32.065002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:54:32.065002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:54:32.065002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:54:32.065002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:54:32.066006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:54:32.066006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:54:32.066006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:54:32.066006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:54:32.066006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:32.066006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:32.066006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:32.066006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:32.066006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:32.066006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:32.066006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:32.066006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:32.066006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:32.066006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:32.066006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:32.066006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:32.066935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:32.066935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:32.066935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:32.066935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:32.066935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:54:32.066935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:54:32.066935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:54:32.066935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:54:32.066935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:54:32.066935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:54:32.066935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 13:54:32.066935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:32.070698 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 464/939] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Jul 11 13:54:31 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:32.073654 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1064b-0e29-4af4-be5d-e3bfcc31125f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:32.085579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:32.187986 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b1064b-0e29-4af4-be5d-e3bfcc31125f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:32.226200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a2838c-e1c6-47b3-a14f-cd0c589070e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:32.227268 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2838c-e1c6-47b3-a14f-cd0c589070e0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:32.227755 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2838c-e1c6-47b3-a14f-cd0c589070e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:32.228216 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a2838c-e1c6-47b3-a14f-cd0c589070e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:32.229185 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2838c-e1c6-47b3-a14f-cd0c589070e0 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:32.229714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2838c-e1c6-47b3-a14f-cd0c589070e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:32.231518 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a2838c-e1c6-47b3-a14f-cd0c589070e0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:32.246485 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 476/940] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Jul 11 13:54:32 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:32.248217 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac7af8-82d2-421b-8ac9-ed67b08cd79d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:32.256008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f672d-ea51-417a-8be3-8756c7f5e4c3 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:32.256323 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f672d-ea51-417a-8be3-8756c7f5e4c3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:32.256588 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d1f672d-ea51-417a-8be3-8756c7f5e4c3 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:32.258290 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 477/941] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:32.263892 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6af79-0ce4-48e6-9b5e-2e7093eedf5c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:32.264231 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6af79-0ce4-48e6-9b5e-2e7093eedf5c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:32.264462 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a6af79-0ce4-48e6-9b5e-2e7093eedf5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:32.285985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-23a6af79-0ce4-48e6-9b5e-2e7093eedf5c None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:32.287452 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-23a6af79-0ce4-48e6-9b5e-2e7093eedf5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:32.376570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:32.381458 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6ac7af8-82d2-421b-8ac9-ed67b08cd79d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:32.385879 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 478/942] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:32 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:54:32.402210 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6af79-0ce4-48e6-9b5e-2e7093eedf5c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:32.420144 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9c1c6-8822-4645-bb7e-b872792472ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:32.421371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9c1c6-8822-4645-bb7e-b872792472ad None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:32.421898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9c1c6-8822-4645-bb7e-b872792472ad None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:32.422444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa9c1c6-8822-4645-bb7e-b872792472ad None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:32.435997 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa9c1c6-8822-4645-bb7e-b872792472ad None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:32.438170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa9c1c6-8822-4645-bb7e-b872792472ad None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:32.446904 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:32.460571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-7aa9c1c6-8822-4645-bb7e-b872792472ad None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:32.460571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:32.460571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:32.460571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:32.460571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:32.460571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:32.460571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:32.460571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:32.460571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:32.460571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:32.460571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:32.460571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:32.460571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:32.460571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:32.460571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:32.461661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:32.461661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:32.461661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:32.461661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:32.461661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:32.461661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:32.461661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:32.461661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:32.461661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:32.461661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:32.461661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:32.461661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:32.461661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:32.461661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:32.461661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:32.461661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:32.461661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:32.462728 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:32.462728 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:32.462728 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:32.462728 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:32.465202 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 465/943] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:32.474353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9c1c6-8822-4645-bb7e-b872792472ad None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:32.525215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a6af79-0ce4-48e6-9b5e-2e7093eedf5c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:32.577100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c0df2-f8f2-4e68-a562-cb7192f74248 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:32.578033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c0df2-f8f2-4e68-a562-cb7192f74248 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:32.578355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c0df2-f8f2-4e68-a562-cb7192f74248 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:32.578642 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc0c0df2-f8f2-4e68-a562-cb7192f74248 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:32.579630 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c0df2-f8f2-4e68-a562-cb7192f74248 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:32.581833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc0c0df2-f8f2-4e68-a562-cb7192f74248 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:32.599409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-dc0c0df2-f8f2-4e68-a562-cb7192f74248 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 13:54:32.599409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:32.599409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:32.599409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:32.599409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:32.599409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:32.599409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:32.599409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:32.599409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 13:54:32.599409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:32.599409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:32.599409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:32.599409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:32.599409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:32.599409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:32.599409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:32.599409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:32.600713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:32.600713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:32.600713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:32.600713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:32.600713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:32.600713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:32.600713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:32.600713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:32.600713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:32.600713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:32.600713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:32.600713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:32.600713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:32.600713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:32.600713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:32.600713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:32.601550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:32.601550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:32.601550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:32.601550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:32.601550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:32.601550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:32.601550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:32.601550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:32.601550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:32.601550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:32.601550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:32.601550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:32.601550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 13:54:32.601550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:32.605344 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa9c1c6-8822-4645-bb7e-b872792472ad None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:32.605952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 479/944] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:32 2022] GET /identity/v3/services/compute => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:32.607868 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c0df2-f8f2-4e68-a562-cb7192f74248 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:32.642694 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6173f0-748e-44b1-9c21-11f2668e2b1e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:32.643512 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6173f0-748e-44b1-9c21-11f2668e2b1e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:32.643901 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6173f0-748e-44b1-9c21-11f2668e2b1e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:32.644245 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc6173f0-748e-44b1-9c21-11f2668e2b1e None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:32.645153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6173f0-748e-44b1-9c21-11f2668e2b1e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:32.645626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6173f0-748e-44b1-9c21-11f2668e2b1e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:32.647150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc6173f0-748e-44b1-9c21-11f2668e2b1e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:32.670155 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 466/945] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:32.672054 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6173f0-748e-44b1-9c21-11f2668e2b1e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:32.688773 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:32.745710 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc0c0df2-f8f2-4e68-a562-cb7192f74248 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:32.747918 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:32.791349 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e876bf-63d9-4be3-b382-269e5767d769 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:32.792031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e876bf-63d9-4be3-b382-269e5767d769 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:32.792313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e876bf-63d9-4be3-b382-269e5767d769 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:32.792620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5e876bf-63d9-4be3-b382-269e5767d769 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:32.793968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e876bf-63d9-4be3-b382-269e5767d769 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:32.794329 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e876bf-63d9-4be3-b382-269e5767d769 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:32.796059 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5e876bf-63d9-4be3-b382-269e5767d769 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:32.806482 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 480/946] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 11 13:54:32 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:32.811137 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e876bf-63d9-4be3-b382-269e5767d769 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:32.812305 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc6173f0-748e-44b1-9c21-11f2668e2b1e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:32.855863 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a28a1-d115-4027-b829-f2fc563c9a05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:32.856685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a28a1-d115-4027-b829-f2fc563c9a05 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:32.856999 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a28a1-d115-4027-b829-f2fc563c9a05 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:32.857279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-769a28a1-d115-4027-b829-f2fc563c9a05 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:32.864960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a28a1-d115-4027-b829-f2fc563c9a05 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:32.866456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-769a28a1-d115-4027-b829-f2fc563c9a05 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:32.882221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-769a28a1-d115-4027-b829-f2fc563c9a05 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 13:54:32.882221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:32.882221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:32.882221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:32.882221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:32.882221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:32.882221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:32.882221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:32.882221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:32.882221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:32.882221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:32.882221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:32.882221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:54:32.882221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:54:32.882221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:54:32.882221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:54:32.883327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:32.883327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:32.883327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:54:32.883327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:32.883327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:54:32.883327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:32.883327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:32.883327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:32.883327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:32.883327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:32.883327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:32.883327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:32.883327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:32.883327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:32.883327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:32.883327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:32.883327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:32.883327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:32.884283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:32.884283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:32.884283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:54:32.884283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:54:32.884283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:54:32.884283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:54:32.884283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:54:32.884283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:54:32.884283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 13:54:32.884283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:32.884283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 467/947] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 11 13:54:32 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:32.891894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a28a1-d115-4027-b829-f2fc563c9a05 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:32.945976 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5e876bf-63d9-4be3-b382-269e5767d769 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:32.984719 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:32.997028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3ccf6-c8e4-4e39-a78e-eca9c1477307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:32.998477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3ccf6-c8e4-4e39-a78e-eca9c1477307 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:32.998768 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3ccf6-c8e4-4e39-a78e-eca9c1477307 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:32.999050 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b3ccf6-c8e4-4e39-a78e-eca9c1477307 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:33.000054 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3ccf6-c8e4-4e39-a78e-eca9c1477307 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:33.000370 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3ccf6-c8e4-4e39-a78e-eca9c1477307 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:33.002169 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b3ccf6-c8e4-4e39-a78e-eca9c1477307 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:33.010478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 481/948] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 11 13:54:32 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:33.022072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-769a28a1-d115-4027-b829-f2fc563c9a05 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:33.047734 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:33.063274 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb402ccd-fa8f-498d-97b2-1afaae05b059 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:33.063957 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb402ccd-fa8f-498d-97b2-1afaae05b059 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:33.064237 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb402ccd-fa8f-498d-97b2-1afaae05b059 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:33.064510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb402ccd-fa8f-498d-97b2-1afaae05b059 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:33.065364 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb402ccd-fa8f-498d-97b2-1afaae05b059 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:33.065670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb402ccd-fa8f-498d-97b2-1afaae05b059 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:33.067142 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb402ccd-fa8f-498d-97b2-1afaae05b059 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:33.092937 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 468/949] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Jul 11 13:54:32 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:33.098943 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3ccf6-c8e4-4e39-a78e-eca9c1477307 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:33.139452 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10719adb-3c7e-4b23-ac27-aed888870e3a None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:33.139790 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10719adb-3c7e-4b23-ac27-aed888870e3a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:33.140022 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10719adb-3c7e-4b23-ac27-aed888870e3a None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:33.142504 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 469/950] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:33.147932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097251e0-2330-4777-b1df-80998c188bf5 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:33.147932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097251e0-2330-4777-b1df-80998c188bf5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:33.147932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097251e0-2330-4777-b1df-80998c188bf5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:33.166773 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-097251e0-2330-4777-b1df-80998c188bf5 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:33.168068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-097251e0-2330-4777-b1df-80998c188bf5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:33.228008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b3ccf6-c8e4-4e39-a78e-eca9c1477307 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:33.259724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 470/951] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:54:33.266388 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21ba37-2b27-411e-bbdc-e725605b36e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:33.267449 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21ba37-2b27-411e-bbdc-e725605b36e2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:33.267972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21ba37-2b27-411e-bbdc-e725605b36e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:33.268454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab21ba37-2b27-411e-bbdc-e725605b36e2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:33.280177 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21ba37-2b27-411e-bbdc-e725605b36e2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:33.281935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab21ba37-2b27-411e-bbdc-e725605b36e2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:33.284978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097251e0-2330-4777-b1df-80998c188bf5 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:33.302000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-ab21ba37-2b27-411e-bbdc-e725605b36e2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:33.302000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:33.302000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:33.302000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:33.302000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:33.302000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:33.302000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:33.302000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:33.302000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:33.302000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:33.302000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:33.302000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:33.302000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:33.302000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:33.302000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:33.302973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:33.302973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:33.302973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:33.302973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:33.302973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:33.302973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:33.302973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:33.302973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:33.302973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:33.302973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:33.302973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:33.302973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:33.302973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:33.302973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:33.302973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:33.302973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:33.302973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:33.303846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:33.303846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:33.303846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:33.303846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:33.306450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 482/952] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:33.313284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21ba37-2b27-411e-bbdc-e725605b36e2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:33.370740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:33.392680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097251e0-2330-4777-b1df-80998c188bf5 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:33.401100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:33.431672 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d8d62-8ded-4a4b-8d84-d5ae31f848ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:33.432576 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d8d62-8ded-4a4b-8d84-d5ae31f848ae None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:33.432933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d8d62-8ded-4a4b-8d84-d5ae31f848ae None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:33.433123 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab21ba37-2b27-411e-bbdc-e725605b36e2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:33.433344 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8d8d62-8ded-4a4b-8d84-d5ae31f848ae None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:33.434161 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d8d62-8ded-4a4b-8d84-d5ae31f848ae None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:33.436374 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8d8d62-8ded-4a4b-8d84-d5ae31f848ae None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:33.449646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-ff8d8d62-8ded-4a4b-8d84-d5ae31f848ae None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 13:54:33.449646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:33.449646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:33.449646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:33.449646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:33.449646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:33.449646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:33.449646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:33.449646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 13:54:33.449646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:33.449646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:33.449646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:33.449646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:33.449646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:33.449646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:33.449646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:33.449646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:33.451001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:33.451001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:33.451001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:33.451001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:33.451001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:33.451001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:33.451001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:33.451001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:33.451001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:33.451001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:33.451001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:33.451001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:33.451001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:33.451001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:33.451001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:33.451001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:33.452278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:33.452278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:33.452278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:33.452278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:33.452278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:33.452278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:33.452278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:33.452278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:33.452278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:33.452278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:33.452278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:33.452278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:33.452278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 13:54:33.452278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:33.458403 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 471/953] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 11 13:54:33 2022] GET /identity/v3/services/object-store => generated 93 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:33.459355 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d8d62-8ded-4a4b-8d84-d5ae31f848ae None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:33.476120 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789b654-ee15-47c3-a200-4c05ee3202fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:33.477109 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789b654-ee15-47c3-a200-4c05ee3202fe None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:33.477580 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789b654-ee15-47c3-a200-4c05ee3202fe None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:33.478139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3789b654-ee15-47c3-a200-4c05ee3202fe None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:33.479124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789b654-ee15-47c3-a200-4c05ee3202fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:33.479701 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789b654-ee15-47c3-a200-4c05ee3202fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:33.481453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3789b654-ee15-47c3-a200-4c05ee3202fe None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:33.502134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 483/954] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:33.503976 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789b654-ee15-47c3-a200-4c05ee3202fe None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:33.579900 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8d8d62-8ded-4a4b-8d84-d5ae31f848ae None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:33.618979 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf3050-ddb4-415b-b1c7-58e90448d99a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:33.619748 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf3050-ddb4-415b-b1c7-58e90448d99a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:33.619965 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf3050-ddb4-415b-b1c7-58e90448d99a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:33.620263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbf3050-ddb4-415b-b1c7-58e90448d99a None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:33.620980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf3050-ddb4-415b-b1c7-58e90448d99a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:33.621294 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf3050-ddb4-415b-b1c7-58e90448d99a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:33.623265 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fbf3050-ddb4-415b-b1c7-58e90448d99a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:33.623661 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3789b654-ee15-47c3-a200-4c05ee3202fe None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:33.626546 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:33.630601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 472/955] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 11 13:54:33 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:33.642421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf3050-ddb4-415b-b1c7-58e90448d99a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:33.663616 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfa632-56b4-4521-8650-bb0524f8d7d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:33.664419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfa632-56b4-4521-8650-bb0524f8d7d9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:33.664747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfa632-56b4-4521-8650-bb0524f8d7d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:33.665025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efdfa632-56b4-4521-8650-bb0524f8d7d9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:33.673336 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfa632-56b4-4521-8650-bb0524f8d7d9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:33.674987 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efdfa632-56b4-4521-8650-bb0524f8d7d9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:33.680526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:33.691232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-efdfa632-56b4-4521-8650-bb0524f8d7d9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:33.691232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:33.691232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:33.691232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:33.691232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:33.691232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:33.691232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:33.691232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:33.691232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:33.691232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:33.691232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:33.691232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:33.691232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:54:33.691232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:54:33.691232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:54:33.692276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:54:33.692276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:33.692276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:33.692276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:33.692276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:33.692276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:33.692276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:33.692276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:33.692276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:33.692276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:33.692276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:33.692276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:33.692276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:33.692276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:33.692276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:33.692276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:33.692276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:33.693148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:54:33.693148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:54:33.693148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:54:33.693148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:54:33.693148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:54:33.693148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:54:33.693148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:33.693148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:33.697429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 484/956] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:33 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:33.698804 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfa632-56b4-4521-8650-bb0524f8d7d9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:33.757831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbf3050-ddb4-415b-b1c7-58e90448d99a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:33.802220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-721d7c23-57b2-4fc8-be5f-4e65a767ac7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:33.802916 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d7c23-57b2-4fc8-be5f-4e65a767ac7f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:33.803197 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d7c23-57b2-4fc8-be5f-4e65a767ac7f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:33.803471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-721d7c23-57b2-4fc8-be5f-4e65a767ac7f None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:33.804183 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d7c23-57b2-4fc8-be5f-4e65a767ac7f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:33.804488 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d7c23-57b2-4fc8-be5f-4e65a767ac7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:33.806112 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-721d7c23-57b2-4fc8-be5f-4e65a767ac7f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:33.810138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efdfa632-56b4-4521-8650-bb0524f8d7d9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:33.811299 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 473/957] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 11 13:54:33 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:33.812513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68926a1-9e52-4621-8564-2a437e03b2da None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:33.812798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68926a1-9e52-4621-8564-2a437e03b2da None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:33.813072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68926a1-9e52-4621-8564-2a437e03b2da None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:33.814296 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 474/958] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:33.816564 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68926a1-9e52-4621-8564-2a437e03b2da None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:33.858266 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2612f89-a8df-45b4-af6f-79d295f345e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:33.859004 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2612f89-a8df-45b4-af6f-79d295f345e0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:33.859299 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2612f89-a8df-45b4-af6f-79d295f345e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:33.859582 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2612f89-a8df-45b4-af6f-79d295f345e0 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:33.860397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2612f89-a8df-45b4-af6f-79d295f345e0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:33.860712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2612f89-a8df-45b4-af6f-79d295f345e0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:33.862135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2612f89-a8df-45b4-af6f-79d295f345e0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:33.875948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 485/959] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:54:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:33.877268 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f71626-2212-4b84-9de3-bbc163e4a5f6 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:33.877558 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f71626-2212-4b84-9de3-bbc163e4a5f6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:33.877838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f71626-2212-4b84-9de3-bbc163e4a5f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:33.896926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-04f71626-2212-4b84-9de3-bbc163e4a5f6 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:33.898159 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-04f71626-2212-4b84-9de3-bbc163e4a5f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:33.921294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:33.954582 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:33.962443 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68926a1-9e52-4621-8564-2a437e03b2da None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:33.994332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 486/960] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:33 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:33.994332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f71626-2212-4b84-9de3-bbc163e4a5f6 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:34.010450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6966ea8-3567-4fd7-ace0-54b8666a3352 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:34.011140 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6966ea8-3567-4fd7-ace0-54b8666a3352 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:34.011640 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6966ea8-3567-4fd7-ace0-54b8666a3352 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:34.012128 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6966ea8-3567-4fd7-ace0-54b8666a3352 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:34.012907 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6966ea8-3567-4fd7-ace0-54b8666a3352 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:34.014414 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6966ea8-3567-4fd7-ace0-54b8666a3352 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:34.036391 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 475/961] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Jul 11 13:54:33 2022] POST /identity/v3/endpoints => generated 349 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:54:34.037584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6966ea8-3567-4fd7-ace0-54b8666a3352 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:34.152614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f71626-2212-4b84-9de3-bbc163e4a5f6 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:34.209816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073097b3-74df-4c2e-9e31-346218ea238f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:34.210882 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073097b3-74df-4c2e-9e31-346218ea238f None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:34.211178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073097b3-74df-4c2e-9e31-346218ea238f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:34.211438 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073097b3-74df-4c2e-9e31-346218ea238f None admin] PATH_INFO: `/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/52076f59f68f4d77a0dadbf7a98d2b57/roles/836753c7d9b44fde817a38c1566d0659` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:34.213727 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073097b3-74df-4c2e-9e31-346218ea238f None admin] RBAC: Authorizing `identity:create_grant(project_id=08ee65f8b338491f80e7beac5e316b23, user_id=52076f59f68f4d77a0dadbf7a98d2b57, role_id=836753c7d9b44fde817a38c1566d0659)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:34.215268 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073097b3-74df-4c2e-9e31-346218ea238f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:34.234168 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6966ea8-3567-4fd7-ace0-54b8666a3352 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:34.239673 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 487/962] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:54:33 2022] PUT /identity/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/52076f59f68f4d77a0dadbf7a98d2b57/roles/836753c7d9b44fde817a38c1566d0659 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:54:34.284197 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5fcce-8782-401a-9e54-9d1f5b5a172d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:34.285719 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5fcce-8782-401a-9e54-9d1f5b5a172d None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:34.286222 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5fcce-8782-401a-9e54-9d1f5b5a172d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:34.286724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9af5fcce-8782-401a-9e54-9d1f5b5a172d None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:34.287525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5fcce-8782-401a-9e54-9d1f5b5a172d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:34.289103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9af5fcce-8782-401a-9e54-9d1f5b5a172d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:34.290570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: WARNING keystone.server.flask.common [None req-9af5fcce-8782-401a-9e54-9d1f5b5a172d None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 11 13:54:34.305768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.sql.core [None req-9af5fcce-8782-401a-9e54-9d1f5b5a172d None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 11 13:54:34.305768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 11 13:54:34.305768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [parameters: {'user_id': 'a45d3ed1a1ee4bcea71b8c2d8c53d850', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 11 13:54:34.305768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80247) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 11 13:54:34.323898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-9af5fcce-8782-401a-9e54-9d1f5b5a172d None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 11 13:54:34.323898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:34.323898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 13:54:34.323898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 13:54:34.323898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 13:54:34.323898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 13:54:34.323898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 13:54:34.323898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 13:54:34.323898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 13:54:34.323898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application conn.query(q) Jul 11 13:54:34.323898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 13:54:34.323898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 13:54:34.323898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 13:54:34.323898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application result.read() Jul 11 13:54:34.323898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 13:54:34.324897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 13:54:34.324897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 13:54:34.324897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 13:54:34.324897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 13:54:34.324897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 13:54:34.324897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 13:54:34.324897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 13:54:34.324897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 11 13:54:34.324897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:34.324897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 11 13:54:34.324897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:34.324897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:34.324897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 11 13:54:34.324897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 11 13:54:34.324897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 11 13:54:34.324897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 11 13:54:34.324897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 13:54:34.324897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application next(self.gen) Jul 11 13:54:34.326633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 11 13:54:34.326633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application yield resource Jul 11 13:54:34.326633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 11 13:54:34.326633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application next(self.gen) Jul 11 13:54:34.326633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 11 13:54:34.326633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.session.rollback() Jul 11 13:54:34.326633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 11 13:54:34.326633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.force_reraise() Jul 11 13:54:34.326633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 11 13:54:34.326633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise self.value Jul 11 13:54:34.326633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 11 13:54:34.326633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 11 13:54:34.326633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 11 13:54:34.326633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application session.commit() Jul 11 13:54:34.326633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 11 13:54:34.326633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 11 13:54:34.326633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 11 13:54:34.327880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._prepare_impl() Jul 11 13:54:34.327880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 11 13:54:34.327880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.session.flush() Jul 11 13:54:34.327880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 11 13:54:34.327880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._flush(objects) Jul 11 13:54:34.327880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 11 13:54:34.327880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 11 13:54:34.327880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 11 13:54:34.327880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application compat.raise_( Jul 11 13:54:34.327880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 13:54:34.327880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception Jul 11 13:54:34.327880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 11 13:54:34.327880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application flush_context.execute() Jul 11 13:54:34.327880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 11 13:54:34.327880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rec.execute(self) Jul 11 13:54:34.327880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 11 13:54:34.327880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 11 13:54:34.328769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 11 13:54:34.328769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 11 13:54:34.328769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 11 13:54:34.328769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 11 13:54:34.328769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 11 13:54:34.328769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 11 13:54:34.328769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 11 13:54:34.328769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 11 13:54:34.328769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 11 13:54:34.328769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 11 13:54:34.328769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 11 13:54:34.328769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 11 13:54:34.328769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 11 13:54:34.328769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 11 13:54:34.328769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 11 13:54:34.328769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception Jul 11 13:54:34.329619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 11 13:54:34.329619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 11 13:54:34.329619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 11 13:54:34.329619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 11 13:54:34.329619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 11 13:54:34.329619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application result = self._query(query) Jul 11 13:54:34.329619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 11 13:54:34.329619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application conn.query(q) Jul 11 13:54:34.329619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 11 13:54:34.329619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 11 13:54:34.329619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 11 13:54:34.329619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application result.read() Jul 11 13:54:34.329619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 11 13:54:34.329619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 11 13:54:34.329619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 11 13:54:34.329619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 11 13:54:34.336328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 11 13:54:34.336328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 11 13:54:34.336328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 11 13:54:34.336328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 11 13:54:34.336328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 11 13:54:34.336328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 11 13:54:34.336328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [parameters: {'user_id': 'a45d3ed1a1ee4bcea71b8c2d8c53d850', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 11 13:54:34.336328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 11 13:54:34.336328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:34.336328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:34.336328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:34.336328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:34.336328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:34.336328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:34.336328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:34.337775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:34.337775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:34.337775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:34.337775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:34.337775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:34.337775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:34.337775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:34.337775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 11 13:54:34.337775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 11 13:54:34.337775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:34.337775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:34.337775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:54:34.337775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:34.337775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:54:34.337775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:34.337775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 11 13:54:34.337775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 11 13:54:34.338980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 11 13:54:34.338980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 11 13:54:34.338980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 11 13:54:34.338980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 11 13:54:34.338980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 11 13:54:34.338980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:34.338980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 476/963] 10.222.0.49 () {60 vars in 1131 bytes} [Mon Jul 11 13:54:34 2022] POST /identity/v3/users => generated 160 bytes in 288 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:34.339473 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073097b3-74df-4c2e-9e31-346218ea238f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:34.351194 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:34.487184 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073097b3-74df-4c2e-9e31-346218ea238f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:34.548700 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe6c1192-d847-47ee-87f3-d2ee34c68daf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:34.550475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c1192-d847-47ee-87f3-d2ee34c68daf None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:34.551003 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c1192-d847-47ee-87f3-d2ee34c68daf None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:34.551581 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe6c1192-d847-47ee-87f3-d2ee34c68daf None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:34.565888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c1192-d847-47ee-87f3-d2ee34c68daf None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:34.567513 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe6c1192-d847-47ee-87f3-d2ee34c68daf None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:34.584475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-fe6c1192-d847-47ee-87f3-d2ee34c68daf None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 13:54:34.584475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:34.584475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:34.584475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:34.584475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:34.584475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:34.584475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:34.584475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:34.584475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:34.584475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:34.584475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:34.584475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:34.584475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:54:34.584475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:54:34.584475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:54:34.584475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:54:34.588372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:34.588372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:34.588372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:54:34.588372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:34.588372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:54:34.588372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:34.588372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:34.588372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:34.588372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:34.588372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:34.588372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:34.588372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:34.588372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:34.588372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:34.588372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:34.588372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:34.588372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:34.588372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:34.596041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:34.596041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:34.596041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:54:34.596041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:54:34.596041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:54:34.596041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:54:34.596041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:54:34.596041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:54:34.596041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 11 13:54:34.596041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:34.596041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 488/964] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:54:34 2022] GET /identity/v3/users/cinder => generated 84 bytes in 254 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:34.598739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5fcce-8782-401a-9e54-9d1f5b5a172d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:34.616624 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:34.736075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9af5fcce-8782-401a-9e54-9d1f5b5a172d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:34.790061 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42d222b6-6e7a-4faf-bd74-abc8bbfcc4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:34.791052 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d222b6-6e7a-4faf-bd74-abc8bbfcc4ce None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:34.791343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d222b6-6e7a-4faf-bd74-abc8bbfcc4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:34.791623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42d222b6-6e7a-4faf-bd74-abc8bbfcc4ce None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:34.792715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d222b6-6e7a-4faf-bd74-abc8bbfcc4ce None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:34.793034 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d222b6-6e7a-4faf-bd74-abc8bbfcc4ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:34.795560 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42d222b6-6e7a-4faf-bd74-abc8bbfcc4ce None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:34.840459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 477/965] 10.222.0.49 () {56 vars in 1101 bytes} [Mon Jul 11 13:54:34 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:35.312583 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee9def-dfd7-4b32-b94f-20f51c082919 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:35.313126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee9def-dfd7-4b32-b94f-20f51c082919 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:35.313126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ee9def-dfd7-4b32-b94f-20f51c082919 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:35.314639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 489/966] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:54:35.320897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa585ea-52a1-4700-806f-ec392d960471 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:35.321519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa585ea-52a1-4700-806f-ec392d960471 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:35.321519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fa585ea-52a1-4700-806f-ec392d960471 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:35.346556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3fa585ea-52a1-4700-806f-ec392d960471 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:35.348068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3fa585ea-52a1-4700-806f-ec392d960471 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:35.423836 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 478/967] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:35 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:54:35.447089 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee9def-dfd7-4b32-b94f-20f51c082919 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:35.528874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:35.575406 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ee9def-dfd7-4b32-b94f-20f51c082919 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:35.617238 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd408b11-7ce5-4229-a700-23fb557d896a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:35.618029 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd408b11-7ce5-4229-a700-23fb557d896a None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:35.618418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd408b11-7ce5-4229-a700-23fb557d896a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:35.618702 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd408b11-7ce5-4229-a700-23fb557d896a None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:35.619852 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd408b11-7ce5-4229-a700-23fb557d896a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:35.621300 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd408b11-7ce5-4229-a700-23fb557d896a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:35.644992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 490/968] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 11 13:54:35 2022] POST /identity/v3/services => generated 242 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:54:36.288800 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc55d8f-1555-4e17-bcbf-9e33ab3ad5f1 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:36.289135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc55d8f-1555-4e17-bcbf-9e33ab3ad5f1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:36.289360 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc55d8f-1555-4e17-bcbf-9e33ab3ad5f1 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:36.291083 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 479/969] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:36.297408 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a45a9-b524-49ab-9e91-bf0860650925 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:36.297712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a45a9-b524-49ab-9e91-bf0860650925 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:36.297982 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8a45a9-b524-49ab-9e91-bf0860650925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:36.320254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ef8a45a9-b524-49ab-9e91-bf0860650925 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:36.322348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ef8a45a9-b524-49ab-9e91-bf0860650925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:36.405630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a1f2e-5de7-4a5f-a573-e66a0492dd25 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:36.406195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a1f2e-5de7-4a5f-a573-e66a0492dd25 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:36.406195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c76a1f2e-5de7-4a5f-a573-e66a0492dd25 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:36.406997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 491/970] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:36.407761 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 480/971] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:54:36.413546 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c85e16-89b4-47dd-9e1f-f067b7d2b4d0 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:36.413824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c85e16-89b4-47dd-9e1f-f067b7d2b4d0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:36.414107 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30c85e16-89b4-47dd-9e1f-f067b7d2b4d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:36.431210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a1f2e-5de7-4a5f-a573-e66a0492dd25 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:36.432363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-30c85e16-89b4-47dd-9e1f-f067b7d2b4d0 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:36.433568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-30c85e16-89b4-47dd-9e1f-f067b7d2b4d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:36.436553 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:36.493329 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:36.515597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 492/972] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:36 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:36.538962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c85e16-89b4-47dd-9e1f-f067b7d2b4d0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:36.547150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c76a1f2e-5de7-4a5f-a573-e66a0492dd25 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:36.597344 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79142e0b-b705-4aca-9458-9c182023c420 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:36.598709 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79142e0b-b705-4aca-9458-9c182023c420 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:36.599221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79142e0b-b705-4aca-9458-9c182023c420 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:36.599704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79142e0b-b705-4aca-9458-9c182023c420 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:36.600525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79142e0b-b705-4aca-9458-9c182023c420 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:36.602073 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79142e0b-b705-4aca-9458-9c182023c420 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:36.618814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-79142e0b-b705-4aca-9458-9c182023c420 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 13:54:36.618814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:36.618814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:36.618814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:36.618814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:36.618814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:36.618814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:36.618814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:36.618814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 13:54:36.618814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:36.618814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:36.618814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:36.618814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:36.618814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:36.618814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:36.618814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:36.618814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:36.620098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:36.620098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:36.620098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:36.620098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:36.620098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:36.620098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:36.620098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:36.620098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:36.620098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:36.620098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:36.620098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:36.620098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:36.620098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:36.620098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:36.620098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:36.620098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:36.621118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:36.621118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:36.621118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:36.621118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:36.621118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:36.621118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:36.621118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:36.621118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:36.621118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:36.621118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:36.621118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:36.621118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:36.621118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 13:54:36.621118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:36.624276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 481/973] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 11 13:54:36 2022] GET /identity/v3/services/object-store => generated 93 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:36.630323 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79142e0b-b705-4aca-9458-9c182023c420 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:36.666151 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30c85e16-89b4-47dd-9e1f-f067b7d2b4d0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:36.706578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367518a-b251-40d2-b042-ac21d554a6b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:36.707691 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367518a-b251-40d2-b042-ac21d554a6b3 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:36.708139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367518a-b251-40d2-b042-ac21d554a6b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:36.708578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4367518a-b251-40d2-b042-ac21d554a6b3 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:36.709923 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:36.723180 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4367518a-b251-40d2-b042-ac21d554a6b3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:54:36.732298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-4367518a-b251-40d2-b042-ac21d554a6b3 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 13:54:36.732298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:36.732298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:36.732298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:36.732298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:36.732298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:36.732298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:36.732298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:36.732298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:36.732298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:36.732298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:36.732298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:36.732298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:54:36.732298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:54:36.732298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:54:36.732298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:54:36.733380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:54:36.733380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:54:36.733380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:54:36.733380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:54:36.733380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:36.733380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:36.733380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:36.733380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:36.733380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:36.733380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:36.733380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:36.733380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:36.733380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:36.733380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:36.733380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:36.733380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:36.734350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:36.734350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:36.734350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:36.734350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:36.734350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:54:36.734350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:54:36.734350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:54:36.734350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:54:36.734350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:54:36.734350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:54:36.734350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 13:54:36.734350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:36.734350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 493/974] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Jul 11 13:54:36 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:36.741153 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367518a-b251-40d2-b042-ac21d554a6b3 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:36.756576 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79142e0b-b705-4aca-9458-9c182023c420 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:36.796965 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb0d9d-0d88-458e-80ee-b83ab8844397 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:36.797783 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb0d9d-0d88-458e-80ee-b83ab8844397 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:36.798053 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb0d9d-0d88-458e-80ee-b83ab8844397 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:36.798358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9feb0d9d-0d88-458e-80ee-b83ab8844397 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:36.799123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb0d9d-0d88-458e-80ee-b83ab8844397 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:36.799434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb0d9d-0d88-458e-80ee-b83ab8844397 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:36.800049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:36.801044 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9feb0d9d-0d88-458e-80ee-b83ab8844397 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:36.807423 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 482/975] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 11 13:54:36 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:36.813108 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb0d9d-0d88-458e-80ee-b83ab8844397 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:36.866063 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4367518a-b251-40d2-b042-ac21d554a6b3 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:36.906801 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcd14f-8803-4dab-8e22-b39cb96634be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:36.907755 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcd14f-8803-4dab-8e22-b39cb96634be None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:36.908060 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcd14f-8803-4dab-8e22-b39cb96634be None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:36.908668 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6bcd14f-8803-4dab-8e22-b39cb96634be None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:36.909478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcd14f-8803-4dab-8e22-b39cb96634be None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:36.909801 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcd14f-8803-4dab-8e22-b39cb96634be None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:36.911233 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6bcd14f-8803-4dab-8e22-b39cb96634be None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:36.920904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 494/976] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Jul 11 13:54:36 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:36.926341 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcd14f-8803-4dab-8e22-b39cb96634be None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:36.937862 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9feb0d9d-0d88-458e-80ee-b83ab8844397 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:36.977284 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:36.983582 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d60d9e-7995-422b-a619-07f319527ad1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:36.984274 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d60d9e-7995-422b-a619-07f319527ad1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:36.984555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d60d9e-7995-422b-a619-07f319527ad1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:36.984829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15d60d9e-7995-422b-a619-07f319527ad1 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:36.995499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d60d9e-7995-422b-a619-07f319527ad1 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:36.995813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d60d9e-7995-422b-a619-07f319527ad1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:36.997191 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15d60d9e-7995-422b-a619-07f319527ad1 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:37.012094 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 483/977] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 11 13:54:36 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:37.013483 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d60d9e-7995-422b-a619-07f319527ad1 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:37.051674 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6bcd14f-8803-4dab-8e22-b39cb96634be None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:37.061896 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:37.094852 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656d0e4-a055-4cea-9334-ea33c244f9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:37.095656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656d0e4-a055-4cea-9334-ea33c244f9ed None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:37.095947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656d0e4-a055-4cea-9334-ea33c244f9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:37.096231 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1656d0e4-a055-4cea-9334-ea33c244f9ed None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:37.108909 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656d0e4-a055-4cea-9334-ea33c244f9ed None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:37.110598 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1656d0e4-a055-4cea-9334-ea33c244f9ed None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:37.126378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15d60d9e-7995-422b-a619-07f319527ad1 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:37.131288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-1656d0e4-a055-4cea-9334-ea33c244f9ed None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:37.131288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:37.131288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:37.131288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:37.131288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:37.131288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:37.131288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:37.131288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:37.131288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:37.131288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:37.131288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:37.131288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:37.131288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:37.131288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:37.131288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:37.132140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:37.132140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:37.132140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:37.132140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:37.132140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:37.132140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:37.132140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:37.132140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:37.132140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:37.132140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:37.132140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:37.132140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:37.132140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:37.132140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:37.132140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:37.132140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:37.132140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:37.133013 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:37.133013 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:37.133013 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:37.133013 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:37.139992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 495/978] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:37.140582 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656d0e4-a055-4cea-9334-ea33c244f9ed None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:37.168559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2c0c3-9db4-4539-b1b5-da1119eb1034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:37.169355 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2c0c3-9db4-4539-b1b5-da1119eb1034 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:37.169707 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2c0c3-9db4-4539-b1b5-da1119eb1034 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:37.170068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdd2c0c3-9db4-4539-b1b5-da1119eb1034 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:37.171068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2c0c3-9db4-4539-b1b5-da1119eb1034 None admin] RBAC: Adding query filter params (interface=admin, service_id=11e9e9396b5642a0867ced76b22701e5, region_id=RegionOne) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:37.171332 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2c0c3-9db4-4539-b1b5-da1119eb1034 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:37.173169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdd2c0c3-9db4-4539-b1b5-da1119eb1034 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:37.183541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 484/979] 10.222.0.49 () {56 vars in 1249 bytes} [Mon Jul 11 13:54:37 2022] GET /identity/v3/endpoints?service_id=11e9e9396b5642a0867ced76b22701e5&interface=admin®ion_id=RegionOne => generated 193 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:37.184348 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2c0c3-9db4-4539-b1b5-da1119eb1034 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:37.246072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:37.252305 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1656d0e4-a055-4cea-9334-ea33c244f9ed None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:37.295652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc88c30-2e2c-41e6-9e66-75c0b3eb5327 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:37.296329 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc88c30-2e2c-41e6-9e66-75c0b3eb5327 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:37.296618 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc88c30-2e2c-41e6-9e66-75c0b3eb5327 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:37.296899 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dc88c30-2e2c-41e6-9e66-75c0b3eb5327 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:37.297631 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc88c30-2e2c-41e6-9e66-75c0b3eb5327 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:37.297943 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc88c30-2e2c-41e6-9e66-75c0b3eb5327 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:37.299460 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dc88c30-2e2c-41e6-9e66-75c0b3eb5327 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:37.318293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdd2c0c3-9db4-4539-b1b5-da1119eb1034 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:37.318537 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 496/980] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:37.319374 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc88c30-2e2c-41e6-9e66-75c0b3eb5327 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:37.344988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:37.357071 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88668467-bd6e-4f76-973d-eec56d4d1581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:37.357736 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88668467-bd6e-4f76-973d-eec56d4d1581 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:37.358019 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88668467-bd6e-4f76-973d-eec56d4d1581 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:37.358291 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88668467-bd6e-4f76-973d-eec56d4d1581 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:37.359001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88668467-bd6e-4f76-973d-eec56d4d1581 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:37.360363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88668467-bd6e-4f76-973d-eec56d4d1581 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:37.374336 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 485/981] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Jul 11 13:54:37 2022] GET /identity/v3/services => generated 1727 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 13:54:37.442740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dc88c30-2e2c-41e6-9e66-75c0b3eb5327 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:37.480279 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963207b7-3953-4abd-938a-d8952761275e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:37.481112 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963207b7-3953-4abd-938a-d8952761275e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:37.481400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963207b7-3953-4abd-938a-d8952761275e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:37.481758 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963207b7-3953-4abd-938a-d8952761275e None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:37.490130 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963207b7-3953-4abd-938a-d8952761275e None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:37.491712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-963207b7-3953-4abd-938a-d8952761275e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:37.508718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-963207b7-3953-4abd-938a-d8952761275e None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 13:54:37.508718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:37.508718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:37.508718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:37.508718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:37.508718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:37.508718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:37.508718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:37.508718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:37.508718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:37.508718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:37.508718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:37.508718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:54:37.508718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:54:37.508718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:54:37.508718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:54:37.509881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:37.509881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:37.509881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:54:37.509881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:37.509881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:54:37.509881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:37.509881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:37.509881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:37.509881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:37.509881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:37.509881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:37.509881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:37.509881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:37.509881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:37.509881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:37.509881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:37.509881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:37.509881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:37.510878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:37.510878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:37.510878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:54:37.510878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:54:37.510878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:54:37.510878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:54:37.510878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:54:37.510878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:54:37.510878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 11 13:54:37.510878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:37.510878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 497/982] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 11 13:54:37 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:37.518324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88668467-bd6e-4f76-973d-eec56d4d1581 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:37.622987 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88668467-bd6e-4f76-973d-eec56d4d1581 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:37.647314 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:37.661340 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4b71d59-bb58-4d6e-90f2-88f528c66053 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:37.662026 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b71d59-bb58-4d6e-90f2-88f528c66053 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:37.662331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b71d59-bb58-4d6e-90f2-88f528c66053 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:37.662619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b71d59-bb58-4d6e-90f2-88f528c66053 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:37.663470 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b71d59-bb58-4d6e-90f2-88f528c66053 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:37.663785 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b71d59-bb58-4d6e-90f2-88f528c66053 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:37.665212 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4b71d59-bb58-4d6e-90f2-88f528c66053 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:37.683441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 486/983] 10.222.0.49 () {56 vars in 1149 bytes} [Mon Jul 11 13:54:37 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:37.689362 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963207b7-3953-4abd-938a-d8952761275e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:37.736440 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1151c-b7af-41f4-bc9f-38f9e69e17b6 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:37.736725 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1151c-b7af-41f4-bc9f-38f9e69e17b6 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:37.736959 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b1151c-b7af-41f4-bc9f-38f9e69e17b6 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:37.738669 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 487/984] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:54:37.744657 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbec0be-f84c-4946-a80e-6c0ceef694c2 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:37.744944 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbec0be-f84c-4946-a80e-6c0ceef694c2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:37.745220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbec0be-f84c-4946-a80e-6c0ceef694c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:37.762282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-bdbec0be-f84c-4946-a80e-6c0ceef694c2 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:37.764109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bdbec0be-f84c-4946-a80e-6c0ceef694c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:37.789084 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:37.816053 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-963207b7-3953-4abd-938a-d8952761275e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:37.842255 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 488/985] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:37 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:54:37.858945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7c225-c79e-4fff-ba19-b2d9c128e286 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:37.859609 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7c225-c79e-4fff-ba19-b2d9c128e286 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:37.859897 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7c225-c79e-4fff-ba19-b2d9c128e286 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:37.860177 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5b7c225-c79e-4fff-ba19-b2d9c128e286 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:37.865619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbec0be-f84c-4946-a80e-6c0ceef694c2 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:37.936650 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7c225-c79e-4fff-ba19-b2d9c128e286 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:37.938772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5b7c225-c79e-4fff-ba19-b2d9c128e286 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:37.969602 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbec0be-f84c-4946-a80e-6c0ceef694c2 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:37.984331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:38.012070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662c0ab-481e-4ccd-b17a-5dc021e7467c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:38.012759 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662c0ab-481e-4ccd-b17a-5dc021e7467c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:38.013087 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662c0ab-481e-4ccd-b17a-5dc021e7467c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:38.013365 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6662c0ab-481e-4ccd-b17a-5dc021e7467c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:38.014036 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6662c0ab-481e-4ccd-b17a-5dc021e7467c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:38.015472 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6662c0ab-481e-4ccd-b17a-5dc021e7467c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:38.029057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-6662c0ab-481e-4ccd-b17a-5dc021e7467c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 13:54:38.029057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:38.029057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:38.029057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:38.029057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:38.029057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:38.029057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:38.029057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:38.029057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 13:54:38.029057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:38.029057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:38.029057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:38.029057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:38.029057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:38.029057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:38.029057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:38.029057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:38.030444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:38.030444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:38.030444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:38.030444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:38.030444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:38.030444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:38.030444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:38.030444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:38.030444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:38.030444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:38.030444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:38.030444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:38.030444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:38.030444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:38.030444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:38.030444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:38.031461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:38.031461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:38.031461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:38.031461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:38.031461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:38.031461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:38.031461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:38.031461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:38.031461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:38.031461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:38.031461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:38.031461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:38.031461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 13:54:38.031461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:38.032383 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:38.034558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 489/986] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:37 2022] GET /identity/v3/services/compute => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:38.035635 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662c0ab-481e-4ccd-b17a-5dc021e7467c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:38.060497 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 498/987] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:54:37 2022] GET /identity/v3/auth/tokens => generated 1560 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 13:54:38.066006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7c225-c79e-4fff-ba19-b2d9c128e286 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:38.157645 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6662c0ab-481e-4ccd-b17a-5dc021e7467c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:38.184041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5b7c225-c79e-4fff-ba19-b2d9c128e286 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:38.192860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ec2b2-c78e-4fa3-8ca7-2539cd717a18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:38.193532 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ec2b2-c78e-4fa3-8ca7-2539cd717a18 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:38.193827 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ec2b2-c78e-4fa3-8ca7-2539cd717a18 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:38.194107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f92ec2b2-c78e-4fa3-8ca7-2539cd717a18 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:38.194876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ec2b2-c78e-4fa3-8ca7-2539cd717a18 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:38.195176 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ec2b2-c78e-4fa3-8ca7-2539cd717a18 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:38.196554 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f92ec2b2-c78e-4fa3-8ca7-2539cd717a18 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:38.205989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 490/988] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 11 13:54:38 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:38.207034 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ec2b2-c78e-4fa3-8ca7-2539cd717a18 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:38.226899 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f366025-1965-4f21-8794-9f54ebd06f70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:38.227683 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f366025-1965-4f21-8794-9f54ebd06f70 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:38.227971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f366025-1965-4f21-8794-9f54ebd06f70 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:38.228250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f366025-1965-4f21-8794-9f54ebd06f70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:38.240056 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f366025-1965-4f21-8794-9f54ebd06f70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:38.241500 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f366025-1965-4f21-8794-9f54ebd06f70 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:38.250918 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:38.260803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-5f366025-1965-4f21-8794-9f54ebd06f70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:38.260803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:38.260803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:38.260803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:38.260803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:38.260803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:38.260803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:38.260803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:38.260803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:38.260803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:38.260803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:38.260803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:38.260803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:38.260803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:38.260803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:38.261697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:38.261697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:38.261697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:38.261697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:38.261697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:38.261697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:38.261697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:38.261697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:38.261697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:38.261697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:38.261697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:38.261697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:38.261697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:38.261697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:38.261697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:38.261697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:38.261697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:38.262657 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:38.262657 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:38.262657 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:38.262657 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:38.267002 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 499/989] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:38.267766 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f366025-1965-4f21-8794-9f54ebd06f70 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:38.302109 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:38.314586 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f92ec2b2-c78e-4fa3-8ca7-2539cd717a18 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:38.349989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0a139-d2d7-40d0-b673-9708f03e932a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:38.350698 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0a139-d2d7-40d0-b673-9708f03e932a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:38.350980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0a139-d2d7-40d0-b673-9708f03e932a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:38.351255 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83d0a139-d2d7-40d0-b673-9708f03e932a None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:38.351963 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d0a139-d2d7-40d0-b673-9708f03e932a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:38.352268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d0a139-d2d7-40d0-b673-9708f03e932a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:38.353657 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83d0a139-d2d7-40d0-b673-9708f03e932a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:38.359384 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 491/990] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 11 13:54:38 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:38.364818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0a139-d2d7-40d0-b673-9708f03e932a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:38.377568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f366025-1965-4f21-8794-9f54ebd06f70 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:38.414272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931ad8c-9b93-43a1-9b10-c6a3c63d2613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:38.414968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931ad8c-9b93-43a1-9b10-c6a3c63d2613 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:38.415253 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931ad8c-9b93-43a1-9b10-c6a3c63d2613 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:38.415683 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4931ad8c-9b93-43a1-9b10-c6a3c63d2613 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:38.416783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4931ad8c-9b93-43a1-9b10-c6a3c63d2613 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:38.417101 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4931ad8c-9b93-43a1-9b10-c6a3c63d2613 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:38.418932 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4931ad8c-9b93-43a1-9b10-c6a3c63d2613 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:38.439238 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 500/991] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:38.441600 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931ad8c-9b93-43a1-9b10-c6a3c63d2613 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:38.474236 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83d0a139-d2d7-40d0-b673-9708f03e932a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:38.477660 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:38.514605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657b024-b063-4416-844d-8b7a59f22a01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:38.515185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657b024-b063-4416-844d-8b7a59f22a01 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:38.515474 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657b024-b063-4416-844d-8b7a59f22a01 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:38.515758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2657b024-b063-4416-844d-8b7a59f22a01 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:38.516561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657b024-b063-4416-844d-8b7a59f22a01 None admin] RBAC: Adding query filter params (interface=public, service_id=2b7ca8bf67894951bce78add23aa0ee8, region_id=RegionOne) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:38.516882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657b024-b063-4416-844d-8b7a59f22a01 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:38.518256 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2657b024-b063-4416-844d-8b7a59f22a01 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:38.523965 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 492/992] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Jul 11 13:54:38 2022] GET /identity/v3/endpoints?service_id=2b7ca8bf67894951bce78add23aa0ee8&interface=public®ion_id=RegionOne => generated 194 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:38.529628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657b024-b063-4416-844d-8b7a59f22a01 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:38.549689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:38.559868 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4931ad8c-9b93-43a1-9b10-c6a3c63d2613 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:38.596886 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b389f-1344-4e20-8da1-a51bfd04d12d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:38.597702 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b389f-1344-4e20-8da1-a51bfd04d12d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:38.597991 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b389f-1344-4e20-8da1-a51bfd04d12d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:38.598271 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b40b389f-1344-4e20-8da1-a51bfd04d12d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:38.605925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b389f-1344-4e20-8da1-a51bfd04d12d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:38.607530 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b40b389f-1344-4e20-8da1-a51bfd04d12d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:38.622297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b40b389f-1344-4e20-8da1-a51bfd04d12d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:38.622297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:38.622297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:38.622297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:38.622297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:38.622297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:38.622297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:38.622297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:38.622297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:38.622297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:38.622297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:38.622297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:38.622297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:54:38.622297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:54:38.622297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:54:38.623247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:54:38.623247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:38.623247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:38.623247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:38.623247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:38.623247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:38.623247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:38.623247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:38.623247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:38.623247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:38.623247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:38.623247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:38.623247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:38.623247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:38.623247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:38.623247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:38.623247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:38.624167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:54:38.624167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:54:38.624167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:54:38.624167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:54:38.624167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:54:38.624167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:54:38.624167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:38.624167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:38.624167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 501/993] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:38.629139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b389f-1344-4e20-8da1-a51bfd04d12d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:38.641537 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2657b024-b063-4416-844d-8b7a59f22a01 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:38.678923 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6124be-e4cd-4da6-897e-1e308627542b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:38.679573 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6124be-e4cd-4da6-897e-1e308627542b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:38.680111 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6124be-e4cd-4da6-897e-1e308627542b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:38.680385 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b6124be-e4cd-4da6-897e-1e308627542b None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:38.681095 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6124be-e4cd-4da6-897e-1e308627542b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:38.682485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b6124be-e4cd-4da6-897e-1e308627542b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:38.699407 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 493/994] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Jul 11 13:54:38 2022] GET /identity/v3/services => generated 1727 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 13:54:38.736682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b40b389f-1344-4e20-8da1-a51bfd04d12d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:38.775995 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f341b8-86bf-4139-befb-759ca62d3ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:38.776586 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f341b8-86bf-4139-befb-759ca62d3ebd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:38.776872 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f341b8-86bf-4139-befb-759ca62d3ebd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:38.777158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f341b8-86bf-4139-befb-759ca62d3ebd None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:38.777966 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f341b8-86bf-4139-befb-759ca62d3ebd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:38.778278 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f341b8-86bf-4139-befb-759ca62d3ebd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:38.780350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f341b8-86bf-4139-befb-759ca62d3ebd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:38.798467 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 502/995] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:54:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:38.800059 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6124be-e4cd-4da6-897e-1e308627542b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:38.835998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:38.914288 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b6124be-e4cd-4da6-897e-1e308627542b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:38.949688 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce831ea5-5037-400b-b3ed-bbc8b8f12945 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:38.951552 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce831ea5-5037-400b-b3ed-bbc8b8f12945 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:38.952082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce831ea5-5037-400b-b3ed-bbc8b8f12945 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:38.952590 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce831ea5-5037-400b-b3ed-bbc8b8f12945 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:38.953656 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce831ea5-5037-400b-b3ed-bbc8b8f12945 None admin] RBAC: Adding query filter params (role.id=836753c7d9b44fde817a38c1566d0659, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=52076f59f68f4d77a0dadbf7a98d2b57) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:38.954198 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce831ea5-5037-400b-b3ed-bbc8b8f12945 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:38.955831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce831ea5-5037-400b-b3ed-bbc8b8f12945 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:38.962733 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 494/996] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:54:38 2022] GET /identity/v3/role_assignments?user.id=52076f59f68f4d77a0dadbf7a98d2b57&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=836753c7d9b44fde817a38c1566d0659 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:39.448929 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48fd89-7ff1-420f-a6dc-28c6b6646f4e None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:39.449478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48fd89-7ff1-420f-a6dc-28c6b6646f4e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:39.449478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d48fd89-7ff1-420f-a6dc-28c6b6646f4e None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:39.451241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 503/997] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:39.456692 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6058a5-a809-4a58-80b8-6a60630f22ea None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:39.456983 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6058a5-a809-4a58-80b8-6a60630f22ea None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:39.457261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6058a5-a809-4a58-80b8-6a60630f22ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:39.473716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5c6058a5-a809-4a58-80b8-6a60630f22ea None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:39.474915 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5c6058a5-a809-4a58-80b8-6a60630f22ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:39.545146 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 495/998] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:39 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:39.567503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48fd89-7ff1-420f-a6dc-28c6b6646f4e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:39.583797 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:39.673250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d48fd89-7ff1-420f-a6dc-28c6b6646f4e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:39.707835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46980f52-7028-4a46-8b59-879973c92b65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:39.708554 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46980f52-7028-4a46-8b59-879973c92b65 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:39.708837 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46980f52-7028-4a46-8b59-879973c92b65 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:39.709112 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46980f52-7028-4a46-8b59-879973c92b65 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:39.709788 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46980f52-7028-4a46-8b59-879973c92b65 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:39.711153 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46980f52-7028-4a46-8b59-879973c92b65 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:39.723730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-46980f52-7028-4a46-8b59-879973c92b65 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 13:54:39.723730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:39.723730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:39.723730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:39.723730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:39.723730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:39.723730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:39.723730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:39.723730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 13:54:39.723730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:39.723730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:39.723730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:39.723730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:39.723730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:39.723730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:39.723730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:39.723730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:39.724800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:39.724800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:39.724800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:39.724800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:39.724800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:39.724800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:39.724800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:39.724800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:39.724800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:39.724800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:39.724800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:39.724800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:39.724800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:39.724800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:39.724800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:39.724800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:39.725689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:39.725689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:39.725689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:39.725689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:39.725689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:39.725689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:39.725689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:39.725689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:39.725689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:39.725689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:39.725689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:39.725689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:39.725689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 11 13:54:39.725689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:39.729974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 504/999] 10.222.0.49 () {56 vars in 1126 bytes} [Mon Jul 11 13:54:39 2022] GET /identity/v3/services/object-store => generated 93 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:39.730794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6058a5-a809-4a58-80b8-6a60630f22ea None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:39.761349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:39.838017 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6058a5-a809-4a58-80b8-6a60630f22ea None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:39.873427 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73ff38-a9bb-4c79-803e-6d18a900f155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:39.874101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73ff38-a9bb-4c79-803e-6d18a900f155 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:39.874416 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73ff38-a9bb-4c79-803e-6d18a900f155 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:39.874697 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be73ff38-a9bb-4c79-803e-6d18a900f155 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:39.875416 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73ff38-a9bb-4c79-803e-6d18a900f155 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:39.875727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73ff38-a9bb-4c79-803e-6d18a900f155 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:39.877095 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be73ff38-a9bb-4c79-803e-6d18a900f155 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:39.886615 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 496/1000] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 11 13:54:39 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:39.888141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46980f52-7028-4a46-8b59-879973c92b65 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:39.970828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:39.997739 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46980f52-7028-4a46-8b59-879973c92b65 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:40.038322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ade883a-8dc3-4d2b-8d3d-3a9c2eab5b48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:40.039026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade883a-8dc3-4d2b-8d3d-3a9c2eab5b48 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:40.039318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade883a-8dc3-4d2b-8d3d-3a9c2eab5b48 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:40.039596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ade883a-8dc3-4d2b-8d3d-3a9c2eab5b48 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:40.040325 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade883a-8dc3-4d2b-8d3d-3a9c2eab5b48 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:40.040636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade883a-8dc3-4d2b-8d3d-3a9c2eab5b48 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:40.042025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ade883a-8dc3-4d2b-8d3d-3a9c2eab5b48 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:40.052619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 505/1001] 10.222.0.49 () {56 vars in 1122 bytes} [Mon Jul 11 13:54:39 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:40.058758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73ff38-a9bb-4c79-803e-6d18a900f155 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:40.172062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:40.187508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be73ff38-a9bb-4c79-803e-6d18a900f155 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:40.226981 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7c9c17-c1ef-4214-89d3-d58a2f912a87 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:40.227567 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c9c17-c1ef-4214-89d3-d58a2f912a87 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:40.227852 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c9c17-c1ef-4214-89d3-d58a2f912a87 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:40.228150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7c9c17-c1ef-4214-89d3-d58a2f912a87 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:40.228926 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c9c17-c1ef-4214-89d3-d58a2f912a87 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:40.230299 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7c9c17-c1ef-4214-89d3-d58a2f912a87 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:40.251898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 497/1002] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Jul 11 13:54:40 2022] POST /identity/v3/endpoints => generated 325 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:54:40.747361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59255ae8-d570-407d-ac7b-28bb5ad7565a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:40.747872 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59255ae8-d570-407d-ac7b-28bb5ad7565a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:40.747950 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59255ae8-d570-407d-ac7b-28bb5ad7565a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:40.749341 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 506/1003] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:40.754412 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50382591-66de-456e-b136-59711e79774d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:40.754787 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50382591-66de-456e-b136-59711e79774d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:40.754971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50382591-66de-456e-b136-59711e79774d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:40.771660 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-50382591-66de-456e-b136-59711e79774d None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:40.772832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-50382591-66de-456e-b136-59711e79774d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:40.847287 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 498/1004] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:40 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:40.869425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59255ae8-d570-407d-ac7b-28bb5ad7565a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:40.976575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59255ae8-d570-407d-ac7b-28bb5ad7565a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:41.011198 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f84db7-3fba-4d1b-bb3b-9638d76b3a50 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:41.011846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f84db7-3fba-4d1b-bb3b-9638d76b3a50 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:41.012131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f84db7-3fba-4d1b-bb3b-9638d76b3a50 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:41.012408 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f84db7-3fba-4d1b-bb3b-9638d76b3a50 None admin] PATH_INFO: `/v3/services/compute` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:41.013602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f84db7-3fba-4d1b-bb3b-9638d76b3a50 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:41.015107 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f84db7-3fba-4d1b-bb3b-9638d76b3a50 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:41.027001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-15f84db7-3fba-4d1b-bb3b-9638d76b3a50 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 13:54:41.027001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:41.027001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:41.027001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:41.027001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:41.027001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:41.027001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:41.027001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:41.027001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 13:54:41.027001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:41.027001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:41.027001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:41.027001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:41.027001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:41.027001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:41.027001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:41.027001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:41.027983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:41.027983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:41.027983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:41.027983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:41.027983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:41.027983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:41.027983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:41.027983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:41.027983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:41.027983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:41.027983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:41.027983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:41.027983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:41.027983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:41.027983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:41.027983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:41.029566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:41.029566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:41.029566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:41.029566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:41.029566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:41.029566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:41.029566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:41.029566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:41.029566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:41.029566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:41.029566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:41.029566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:41.029566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 11 13:54:41.029566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:41.029566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 507/1005] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:40 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:41.032406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e61c0-e9ac-4106-ab62-00dd4e7dfd2b None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:41.032700 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e61c0-e9ac-4106-ab62-00dd4e7dfd2b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:41.032982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e9e61c0-e9ac-4106-ab62-00dd4e7dfd2b None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:41.034389 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f84db7-3fba-4d1b-bb3b-9638d76b3a50 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:41.035187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 499/1006] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:41.038391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:41.039571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d1b10-5e4d-4b6a-b592-8dbf588c9c88 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:41.039824 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d1b10-5e4d-4b6a-b592-8dbf588c9c88 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:41.040097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533d1b10-5e4d-4b6a-b592-8dbf588c9c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:41.060056 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-533d1b10-5e4d-4b6a-b592-8dbf588c9c88 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:41.061250 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-533d1b10-5e4d-4b6a-b592-8dbf588c9c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:41.133794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 500/1007] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:41 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:54:41.152423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f84db7-3fba-4d1b-bb3b-9638d76b3a50 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:41.156806 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d1b10-5e4d-4b6a-b592-8dbf588c9c88 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:41.165571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:41.193790 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba2ce5-8680-4e31-925d-a1c2afd6ae46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:41.194491 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba2ce5-8680-4e31-925d-a1c2afd6ae46 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:41.194781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba2ce5-8680-4e31-925d-a1c2afd6ae46 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:41.195057 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ba2ce5-8680-4e31-925d-a1c2afd6ae46 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:41.195769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba2ce5-8680-4e31-925d-a1c2afd6ae46 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:41.196077 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba2ce5-8680-4e31-925d-a1c2afd6ae46 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:41.197446 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4ba2ce5-8680-4e31-925d-a1c2afd6ae46 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:41.202992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 508/1008] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 11 13:54:41 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:41.208521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba2ce5-8680-4e31-925d-a1c2afd6ae46 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:41.267511 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533d1b10-5e4d-4b6a-b592-8dbf588c9c88 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:41.279946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:41.308946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f36b3b-63ba-4480-9060-e2f2b0b3727c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:41.310247 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f36b3b-63ba-4480-9060-e2f2b0b3727c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:41.310560 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f36b3b-63ba-4480-9060-e2f2b0b3727c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:41.310842 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f36b3b-63ba-4480-9060-e2f2b0b3727c None admin] PATH_INFO: `/v3/services/image` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:41.311497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f36b3b-63ba-4480-9060-e2f2b0b3727c None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:41.312871 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f36b3b-63ba-4480-9060-e2f2b0b3727c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:41.316871 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ba2ce5-8680-4e31-925d-a1c2afd6ae46 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:41.324561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-a0f36b3b-63ba-4480-9060-e2f2b0b3727c None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 13:54:41.324561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:41.324561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:41.324561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:41.324561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:41.324561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:41.324561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:41.324561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:41.324561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 13:54:41.324561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:41.324561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:41.324561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:41.324561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:41.324561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:41.324561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:41.324561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:41.324561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:41.326207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:41.326207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:41.326207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:41.326207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:41.326207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:41.326207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:41.326207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:41.326207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:41.326207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:41.326207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:41.326207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:41.326207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:41.326207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:41.326207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:41.326207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:41.326207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:41.327404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:41.327404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:41.327404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:41.327404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:41.327404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:41.327404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:41.327404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:41.327404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:41.327404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:41.327404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:41.327404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:41.327404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:41.327404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 13:54:41.327404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:41.330609 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 501/1009] 10.222.0.49 () {56 vars in 1105 bytes} [Mon Jul 11 13:54:41 2022] GET /identity/v3/services/image => generated 86 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:41.332188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f36b3b-63ba-4480-9060-e2f2b0b3727c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:41.354866 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e96fbf4-c5b7-4668-8170-e888a5917ff2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:41.355889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96fbf4-c5b7-4668-8170-e888a5917ff2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:41.356243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96fbf4-c5b7-4668-8170-e888a5917ff2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:41.356773 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e96fbf4-c5b7-4668-8170-e888a5917ff2 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:41.357557 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e96fbf4-c5b7-4668-8170-e888a5917ff2 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:41.357937 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e96fbf4-c5b7-4668-8170-e888a5917ff2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:41.359397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e96fbf4-c5b7-4668-8170-e888a5917ff2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:41.364780 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 509/1010] 10.222.0.49 () {56 vars in 1112 bytes} [Mon Jul 11 13:54:41 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:41.370119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e96fbf4-c5b7-4668-8170-e888a5917ff2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:41.409000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:41.447082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f36b3b-63ba-4480-9060-e2f2b0b3727c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:41.476811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e96fbf4-c5b7-4668-8170-e888a5917ff2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:41.501139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d11f8-ee34-49a7-8b25-61abfd2be528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:41.501587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d11f8-ee34-49a7-8b25-61abfd2be528 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:41.501963 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d11f8-ee34-49a7-8b25-61abfd2be528 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:41.502247 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-342d11f8-ee34-49a7-8b25-61abfd2be528 None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:41.503035 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d11f8-ee34-49a7-8b25-61abfd2be528 None admin] RBAC: Adding query filter params (name=image) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:41.503350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d11f8-ee34-49a7-8b25-61abfd2be528 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:41.504730 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-342d11f8-ee34-49a7-8b25-61abfd2be528 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:41.509676 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:41.515538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 502/1011] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:41 2022] GET /identity/v3/services?name=image => generated 122 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:41.516513 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da41b2f4-0416-4680-a450-17d1a478ad90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:41.517072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41b2f4-0416-4680-a450-17d1a478ad90 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:41.517356 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41b2f4-0416-4680-a450-17d1a478ad90 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:41.517952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da41b2f4-0416-4680-a450-17d1a478ad90 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:41.518504 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d11f8-ee34-49a7-8b25-61abfd2be528 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:41.518830 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da41b2f4-0416-4680-a450-17d1a478ad90 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:41.520155 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da41b2f4-0416-4680-a450-17d1a478ad90 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:41.546851 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 510/1012] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Jul 11 13:54:41 2022] POST /identity/v3/endpoints => generated 334 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 13:54:41.650008 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-342d11f8-ee34-49a7-8b25-61abfd2be528 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:41.692587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:41.700829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4dbbadc-94ea-4b23-9732-9ae00868f63c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:41.701664 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbbadc-94ea-4b23-9732-9ae00868f63c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:41.702154 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbbadc-94ea-4b23-9732-9ae00868f63c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:41.702503 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4dbbadc-94ea-4b23-9732-9ae00868f63c None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:41.703599 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dbbadc-94ea-4b23-9732-9ae00868f63c None admin] RBAC: Adding query filter params (type=image) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:41.703708 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dbbadc-94ea-4b23-9732-9ae00868f63c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:41.705127 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4dbbadc-94ea-4b23-9732-9ae00868f63c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:41.711732 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 503/1013] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:41 2022] GET /identity/v3/services?type=image => generated 122 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:42.329838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384e733-b3b7-4f6c-ba3e-3f8df41efb2d None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:42.330354 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384e733-b3b7-4f6c-ba3e-3f8df41efb2d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:42.330354 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7384e733-b3b7-4f6c-ba3e-3f8df41efb2d None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:42.331872 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 511/1014] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:42.336861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750d29f-04de-47d9-8b63-6cc2eead104e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:42.337236 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750d29f-04de-47d9-8b63-6cc2eead104e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:42.337423 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6750d29f-04de-47d9-8b63-6cc2eead104e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:42.355571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-6750d29f-04de-47d9-8b63-6cc2eead104e None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:42.356804 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6750d29f-04de-47d9-8b63-6cc2eead104e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:42.430361 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 504/1015] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:42 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:42.453907 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384e733-b3b7-4f6c-ba3e-3f8df41efb2d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:42.557175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7384e733-b3b7-4f6c-ba3e-3f8df41efb2d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:42.592479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc56c22-1b9f-408e-a8a9-8667988d3d4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:42.593220 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc56c22-1b9f-408e-a8a9-8667988d3d4f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:42.593554 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc56c22-1b9f-408e-a8a9-8667988d3d4f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:42.593841 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc56c22-1b9f-408e-a8a9-8667988d3d4f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:42.595058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc56c22-1b9f-408e-a8a9-8667988d3d4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:42.596497 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc56c22-1b9f-408e-a8a9-8667988d3d4f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:42.599346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 512/1016] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:42 2022] GET /identity/v3/domains/default => generated 199 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:42.604179 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750d29f-04de-47d9-8b63-6cc2eead104e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:42.704796 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6750d29f-04de-47d9-8b63-6cc2eead104e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:42.728766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:42.742028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c01b557a-5d09-4920-98b5-e773f27aac0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:42.742731 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b557a-5d09-4920-98b5-e773f27aac0c None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:42.743023 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b557a-5d09-4920-98b5-e773f27aac0c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:42.743320 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c01b557a-5d09-4920-98b5-e773f27aac0c None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:42.744027 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b557a-5d09-4920-98b5-e773f27aac0c None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:42.745465 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c01b557a-5d09-4920-98b5-e773f27aac0c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:42.778565 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 505/1017] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 11 13:54:42 2022] POST /identity/v3/projects => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:54:43.715285 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258e417-205b-4665-9c3c-cb1452c5a192 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:43.716326 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258e417-205b-4665-9c3c-cb1452c5a192 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:43.716418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d258e417-205b-4665-9c3c-cb1452c5a192 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:43.717908 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 513/1018] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:43.722970 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b52ad3-a6ba-4149-93ea-7c377bb656c1 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:43.723372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b52ad3-a6ba-4149-93ea-7c377bb656c1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:43.723533 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b52ad3-a6ba-4149-93ea-7c377bb656c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:43.740989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-53b52ad3-a6ba-4149-93ea-7c377bb656c1 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:43.742180 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-53b52ad3-a6ba-4149-93ea-7c377bb656c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:43.814422 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 506/1019] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:43 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:43.837852 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258e417-205b-4665-9c3c-cb1452c5a192 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:43.884130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01191d33-866b-4477-aaad-87cb1515ad54 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:43.884444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01191d33-866b-4477-aaad-87cb1515ad54 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:43.884696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01191d33-866b-4477-aaad-87cb1515ad54 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:43.890442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 507/1020] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:54:43.892394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77acbd-1a5b-4300-bf33-581ddbce5706 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:43.892676 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77acbd-1a5b-4300-bf33-581ddbce5706 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:43.892950 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc77acbd-1a5b-4300-bf33-581ddbce5706 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:43.901835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:43.914555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-dc77acbd-1a5b-4300-bf33-581ddbce5706 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:43.916577 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dc77acbd-1a5b-4300-bf33-581ddbce5706 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:43.954926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d258e417-205b-4665-9c3c-cb1452c5a192 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:43.992919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7763b5-8f7c-4828-ac1a-c2459eaa5f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:43.993637 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7763b5-8f7c-4828-ac1a-c2459eaa5f54 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:43.993921 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7763b5-8f7c-4828-ac1a-c2459eaa5f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:43.994194 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac7763b5-8f7c-4828-ac1a-c2459eaa5f54 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:44.002609 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 508/1021] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:43 2022] POST /identity/v3/auth/tokens => generated 1863 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:44.005788 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac7763b5-8f7c-4828-ac1a-c2459eaa5f54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:54:44.014677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-ac7763b5-8f7c-4828-ac1a-c2459eaa5f54 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 13:54:44.014677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:44.014677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:44.014677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:44.014677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:44.014677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:44.014677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:44.014677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:44.014677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:44.014677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:44.014677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:44.014677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:44.014677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:54:44.014677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:54:44.014677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:54:44.014677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:54:44.015652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:54:44.015652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:54:44.015652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:54:44.015652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:54:44.015652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:44.015652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:44.015652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:44.015652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:44.015652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:44.015652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:44.015652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:44.015652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:44.015652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:44.015652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:44.015652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:44.015652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:44.016469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:44.016469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:44.016469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:44.016469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:44.016469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:54:44.016469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:54:44.016469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:54:44.016469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:54:44.016469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:54:44.016469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:54:44.016469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 13:54:44.016469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:44.017719 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 514/1022] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Jul 11 13:54:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:44.021846 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77acbd-1a5b-4300-bf33-581ddbce5706 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:44.026844 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7763b5-8f7c-4828-ac1a-c2459eaa5f54 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:44.069162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:44.156912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc77acbd-1a5b-4300-bf33-581ddbce5706 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:44.170641 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac7763b5-8f7c-4828-ac1a-c2459eaa5f54 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:44.188363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:44.195818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b6b45-e1aa-47d7-a4c4-bb3824d78223 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:44.196565 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b6b45-e1aa-47d7-a4c4-bb3824d78223 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:44.196842 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b6b45-e1aa-47d7-a4c4-bb3824d78223 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:44.197193 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-588b6b45-e1aa-47d7-a4c4-bb3824d78223 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:44.198017 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b6b45-e1aa-47d7-a4c4-bb3824d78223 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:44.198416 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b6b45-e1aa-47d7-a4c4-bb3824d78223 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:44.200115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-588b6b45-e1aa-47d7-a4c4-bb3824d78223 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:44.209670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 509/1023] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Jul 11 13:54:44 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:44.210709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbfc1c-a709-49bc-9c8b-0f39ffa351c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:44.211386 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbfc1c-a709-49bc-9c8b-0f39ffa351c1 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:44.211672 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbfc1c-a709-49bc-9c8b-0f39ffa351c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:44.211951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0adbfc1c-a709-49bc-9c8b-0f39ffa351c1 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:44.212748 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbfc1c-a709-49bc-9c8b-0f39ffa351c1 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:44.214208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0adbfc1c-a709-49bc-9c8b-0f39ffa351c1 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:44.215337 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b6b45-e1aa-47d7-a4c4-bb3824d78223 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:44.234933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 515/1024] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 11 13:54:44 2022] POST /identity/v3/services => generated 242 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 13:54:44.331733 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:44.336855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-588b6b45-e1aa-47d7-a4c4-bb3824d78223 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:44.390583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad013f-ae6b-4184-b19f-f11bfb495c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:44.391472 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad013f-ae6b-4184-b19f-f11bfb495c97 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:44.391797 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad013f-ae6b-4184-b19f-f11bfb495c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:44.392100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ad013f-ae6b-4184-b19f-f11bfb495c97 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:44.404091 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad013f-ae6b-4184-b19f-f11bfb495c97 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:44.405636 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ad013f-ae6b-4184-b19f-f11bfb495c97 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:44.424818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-15ad013f-ae6b-4184-b19f-f11bfb495c97 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:44.424818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:44.424818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:44.424818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:44.424818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:44.424818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:44.424818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:44.424818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:44.424818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:44.424818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:44.424818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:44.424818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:44.424818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:44.424818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:44.424818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:44.425767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:44.425767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:44.425767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:44.425767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:44.425767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:44.425767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:44.425767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:44.425767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:44.425767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:44.425767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:44.425767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:44.425767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:44.425767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:44.425767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:44.425767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:44.425767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:44.425767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:44.426717 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:44.426717 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:44.426717 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:44.426717 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:44.426717 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 510/1025] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:44.432117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbfc1c-a709-49bc-9c8b-0f39ffa351c1 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:44.542343 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0adbfc1c-a709-49bc-9c8b-0f39ffa351c1 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:44.579108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f165f3-e28b-451d-8f13-5990b5760544 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:44.579802 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f165f3-e28b-451d-8f13-5990b5760544 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:44.580092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f165f3-e28b-451d-8f13-5990b5760544 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:44.580397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f165f3-e28b-451d-8f13-5990b5760544 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:44.581139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f165f3-e28b-451d-8f13-5990b5760544 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:44.581473 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f165f3-e28b-451d-8f13-5990b5760544 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:44.583007 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f165f3-e28b-451d-8f13-5990b5760544 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:44.596262 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 516/1026] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:44.601944 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad013f-ae6b-4184-b19f-f11bfb495c97 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:44.707526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ad013f-ae6b-4184-b19f-f11bfb495c97 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:44.745176 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf06361f-3a94-4588-86b3-9afe2502e43f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:44.745980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06361f-3a94-4588-86b3-9afe2502e43f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:44.746262 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06361f-3a94-4588-86b3-9afe2502e43f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:44.746581 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf06361f-3a94-4588-86b3-9afe2502e43f None admin] PATH_INFO: `/v3/users/nova` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:44.755082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf06361f-3a94-4588-86b3-9afe2502e43f None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:44.756521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf06361f-3a94-4588-86b3-9afe2502e43f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:44.771338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-bf06361f-3a94-4588-86b3-9afe2502e43f None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 13:54:44.771338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:44.771338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:44.771338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:44.771338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:44.771338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:44.771338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:44.771338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:44.771338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:44.771338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:44.771338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:44.771338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:44.771338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:54:44.771338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:54:44.771338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:54:44.771338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:54:44.772473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:44.772473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:44.772473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:54:44.772473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:44.772473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:54:44.772473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:44.772473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:44.772473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:44.772473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:44.772473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:44.772473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:44.772473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:44.772473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:44.772473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:44.772473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:44.772473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:44.772473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:44.772473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:44.773550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:44.773550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:44.773550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:54:44.773550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:54:44.773550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:54:44.773550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:54:44.773550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:54:44.773550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:54:44.773550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 13:54:44.773550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:44.773550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 511/1027] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Jul 11 13:54:44 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:44.778416 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f165f3-e28b-451d-8f13-5990b5760544 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:44.815314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:44.884319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f165f3-e28b-451d-8f13-5990b5760544 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:44.889388 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a5f7a-7665-4ea2-82b6-26dd926a1c79 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:44.889744 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a5f7a-7665-4ea2-82b6-26dd926a1c79 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:44.890216 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695a5f7a-7665-4ea2-82b6-26dd926a1c79 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:44.891983 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 512/1028] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:54:44.896969 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a88d4-0a5b-4ed2-be68-17455b312e6e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:44.897454 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a88d4-0a5b-4ed2-be68-17455b312e6e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:44.897743 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b99a88d4-0a5b-4ed2-be68-17455b312e6e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:44.910288 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:44.915840 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b99a88d4-0a5b-4ed2-be68-17455b312e6e None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:44.917213 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b99a88d4-0a5b-4ed2-be68-17455b312e6e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:44.922836 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096fcb2-4009-4db9-8306-55a09f7d3629 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:44.923509 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096fcb2-4009-4db9-8306-55a09f7d3629 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:44.923789 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096fcb2-4009-4db9-8306-55a09f7d3629 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:44.924066 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5096fcb2-4009-4db9-8306-55a09f7d3629 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:44.924903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096fcb2-4009-4db9-8306-55a09f7d3629 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:44.925210 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096fcb2-4009-4db9-8306-55a09f7d3629 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:44.926778 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5096fcb2-4009-4db9-8306-55a09f7d3629 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:44.952122 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 517/1029] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Jul 11 13:54:44 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:44.953317 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096fcb2-4009-4db9-8306-55a09f7d3629 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:44.995700 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 513/1030] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:44 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:54:45.021223 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a88d4-0a5b-4ed2-be68-17455b312e6e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:45.066247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:45.076373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5096fcb2-4009-4db9-8306-55a09f7d3629 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:45.111571 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417248f9-57a9-43b4-b285-afe9d27fb47f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:45.112207 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417248f9-57a9-43b4-b285-afe9d27fb47f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:45.112490 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417248f9-57a9-43b4-b285-afe9d27fb47f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:45.112760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417248f9-57a9-43b4-b285-afe9d27fb47f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:45.139303 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b99a88d4-0a5b-4ed2-be68-17455b312e6e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:45.181573 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7adf26-5c99-474f-b3a9-2ff1443658e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:45.182400 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7adf26-5c99-474f-b3a9-2ff1443658e3 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:45.182684 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7adf26-5c99-474f-b3a9-2ff1443658e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:45.183021 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef7adf26-5c99-474f-b3a9-2ff1443658e3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:45.184157 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7adf26-5c99-474f-b3a9-2ff1443658e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:45.186286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef7adf26-5c99-474f-b3a9-2ff1443658e3 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:45.187991 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 514/1031] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:45.193793 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7adf26-5c99-474f-b3a9-2ff1443658e3 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:45.195666 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417248f9-57a9-43b4-b285-afe9d27fb47f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:45.197115 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-417248f9-57a9-43b4-b285-afe9d27fb47f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:45.198687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:45.314510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:45.317834 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef7adf26-5c99-474f-b3a9-2ff1443658e3 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:45.329355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 518/1032] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:54:44 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 13:54:45.334679 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417248f9-57a9-43b4-b285-afe9d27fb47f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:45.361327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d478ece-db2f-4068-8276-b4d91dea40be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:45.362003 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d478ece-db2f-4068-8276-b4d91dea40be None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:45.362283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d478ece-db2f-4068-8276-b4d91dea40be None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:45.362596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d478ece-db2f-4068-8276-b4d91dea40be None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:45.363320 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d478ece-db2f-4068-8276-b4d91dea40be None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:45.364754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d478ece-db2f-4068-8276-b4d91dea40be None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:45.386632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 515/1033] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 11 13:54:45 2022] POST /identity/v3/users => generated 285 bytes in 193 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 13:54:45.462436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417248f9-57a9-43b4-b285-afe9d27fb47f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:45.501249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb1afe-029f-42a8-b6f8-d306df465c7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:45.502081 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb1afe-029f-42a8-b6f8-d306df465c7d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:45.502383 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb1afe-029f-42a8-b6f8-d306df465c7d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:45.502662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1eb1afe-029f-42a8-b6f8-d306df465c7d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:45.514203 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb1afe-029f-42a8-b6f8-d306df465c7d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:45.515720 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1eb1afe-029f-42a8-b6f8-d306df465c7d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:45.535340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-d1eb1afe-029f-42a8-b6f8-d306df465c7d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:45.535340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:45.535340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:45.535340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:45.535340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:45.535340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:45.535340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:45.535340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:45.535340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:45.535340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:45.535340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:45.535340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:45.535340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:45.535340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:45.535340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:45.536320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:45.536320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:45.536320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:45.536320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:45.536320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:45.536320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:45.536320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:45.536320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:45.536320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:45.536320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:45.536320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:45.536320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:45.536320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:45.536320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:45.536320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:45.536320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:45.536320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:45.537289 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:45.537289 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:45.537289 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:45.537289 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:45.542369 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 519/1034] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:45.543368 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d478ece-db2f-4068-8276-b4d91dea40be None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:45.608774 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:45.651729 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d478ece-db2f-4068-8276-b4d91dea40be None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:45.689846 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba6f6a-74a1-4f60-be71-9eede34033cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:45.690548 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba6f6a-74a1-4f60-be71-9eede34033cb None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:45.690834 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba6f6a-74a1-4f60-be71-9eede34033cb None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:45.691109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eba6f6a-74a1-4f60-be71-9eede34033cb None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:45.691837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba6f6a-74a1-4f60-be71-9eede34033cb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:45.692143 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba6f6a-74a1-4f60-be71-9eede34033cb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:45.693534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eba6f6a-74a1-4f60-be71-9eede34033cb None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:45.706202 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 516/1035] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:45.712090 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb1afe-029f-42a8-b6f8-d306df465c7d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:45.762352 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:45.818137 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1eb1afe-029f-42a8-b6f8-d306df465c7d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:45.852727 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f218991-3268-4240-b2ad-3aadca15fe06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:45.853524 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f218991-3268-4240-b2ad-3aadca15fe06 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:45.853809 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f218991-3268-4240-b2ad-3aadca15fe06 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:45.854076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f218991-3268-4240-b2ad-3aadca15fe06 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:45.861842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f218991-3268-4240-b2ad-3aadca15fe06 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:45.863314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f218991-3268-4240-b2ad-3aadca15fe06 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:45.877750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-1f218991-3268-4240-b2ad-3aadca15fe06 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:45.877750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:45.877750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:45.877750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:45.877750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:45.877750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:45.877750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:45.877750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:45.877750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:45.877750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:45.877750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:45.877750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:45.877750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:54:45.877750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:54:45.877750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:54:45.879000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:54:45.879000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:45.879000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:45.879000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:45.879000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:45.879000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:45.879000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:45.879000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:45.879000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:45.879000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:45.879000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:45.879000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:45.879000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:45.879000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:45.879000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:45.879000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:45.879000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:45.879886 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:54:45.879886 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:54:45.879886 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:54:45.879886 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:54:45.879886 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:54:45.879886 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:54:45.879886 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:45.879886 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:45.879886 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 520/1036] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:45.886473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba6f6a-74a1-4f60-be71-9eede34033cb None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:45.993353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eba6f6a-74a1-4f60-be71-9eede34033cb None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:46.014539 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:46.030833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-800798a3-24ca-42cb-ae78-1d9592c46f79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:46.031503 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800798a3-24ca-42cb-ae78-1d9592c46f79 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:46.031790 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800798a3-24ca-42cb-ae78-1d9592c46f79 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:46.032079 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-800798a3-24ca-42cb-ae78-1d9592c46f79 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:46.032878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800798a3-24ca-42cb-ae78-1d9592c46f79 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:46.033194 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800798a3-24ca-42cb-ae78-1d9592c46f79 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:46.034651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-800798a3-24ca-42cb-ae78-1d9592c46f79 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:46.048905 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 517/1037] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:54:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:46.055889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f218991-3268-4240-b2ad-3aadca15fe06 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:46.165163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f218991-3268-4240-b2ad-3aadca15fe06 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:46.173878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:46.202736 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d955bae-4b87-423e-a116-7089b912af20 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:46.203394 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d955bae-4b87-423e-a116-7089b912af20 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:46.203673 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d955bae-4b87-423e-a116-7089b912af20 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:46.203943 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d955bae-4b87-423e-a116-7089b912af20 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:46.204779 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d955bae-4b87-423e-a116-7089b912af20 None admin] RBAC: Adding query filter params (role.id=836753c7d9b44fde817a38c1566d0659, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=ef5dac3f311e43809a46995bab56e780) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:46.205086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d955bae-4b87-423e-a116-7089b912af20 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:46.206539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d955bae-4b87-423e-a116-7089b912af20 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:46.212505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 521/1038] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:54:46 2022] GET /identity/v3/role_assignments?user.id=ef5dac3f311e43809a46995bab56e780&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=836753c7d9b44fde817a38c1566d0659 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:46.348567 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e0ffa-87f7-490f-8cbb-fd98d875df8f None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:46.348942 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e0ffa-87f7-490f-8cbb-fd98d875df8f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:46.349096 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050e0ffa-87f7-490f-8cbb-fd98d875df8f None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:46.350619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 518/1039] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:46.356253 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec9d6b-b034-451a-9eb2-e305052a3fba None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:46.357103 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec9d6b-b034-451a-9eb2-e305052a3fba None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:46.357553 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ec9d6b-b034-451a-9eb2-e305052a3fba None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:46.376083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-33ec9d6b-b034-451a-9eb2-e305052a3fba None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:46.377293 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-33ec9d6b-b034-451a-9eb2-e305052a3fba None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:46.464681 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 522/1040] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:46 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:46.486337 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e0ffa-87f7-490f-8cbb-fd98d875df8f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:46.601888 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050e0ffa-87f7-490f-8cbb-fd98d875df8f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:46.639290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2270d-e827-4576-914c-9cd22286e017 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:46.640143 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2270d-e827-4576-914c-9cd22286e017 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:46.640431 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2270d-e827-4576-914c-9cd22286e017 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:46.640730 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f2270d-e827-4576-914c-9cd22286e017 None admin] PATH_INFO: `/v3/services/image` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:46.641393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2270d-e827-4576-914c-9cd22286e017 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:46.642812 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f2270d-e827-4576-914c-9cd22286e017 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:46.655473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-79f2270d-e827-4576-914c-9cd22286e017 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 13:54:46.655473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:46.655473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:46.655473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:46.655473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:46.655473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:46.655473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:46.655473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:46.655473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 13:54:46.655473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:46.655473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:46.655473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:46.655473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:46.655473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:46.655473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:46.655473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:46.655473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:46.656878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:46.656878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:46.656878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:46.656878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:46.656878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:46.656878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:46.656878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:46.656878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:46.656878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:46.656878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:46.656878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:46.656878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:46.656878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:46.656878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:46.656878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:46.656878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:46.658282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:46.658282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:46.658282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:46.658282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:46.658282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:46.658282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:46.658282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:46.658282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:46.658282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:46.658282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:46.658282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:46.658282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:46.658282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 13:54:46.658282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:46.658282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 519/1041] 10.222.0.49 () {56 vars in 1105 bytes} [Mon Jul 11 13:54:46 2022] GET /identity/v3/services/image => generated 86 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:46.663452 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec9d6b-b034-451a-9eb2-e305052a3fba None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:46.767765 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:46.771748 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ec9d6b-b034-451a-9eb2-e305052a3fba None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:46.809188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd2af7-1509-409c-acd0-9e2cbfb59fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:46.809881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd2af7-1509-409c-acd0-9e2cbfb59fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:46.810169 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd2af7-1509-409c-acd0-9e2cbfb59fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:46.810478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fabd2af7-1509-409c-acd0-9e2cbfb59fe8 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:46.811192 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd2af7-1509-409c-acd0-9e2cbfb59fe8 None admin] RBAC: Adding query filter params (name=image) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:46.811524 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd2af7-1509-409c-acd0-9e2cbfb59fe8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:46.812932 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fabd2af7-1509-409c-acd0-9e2cbfb59fe8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:46.822972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 523/1042] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:46 2022] GET /identity/v3/services?name=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:46.824631 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2270d-e827-4576-914c-9cd22286e017 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:46.873833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:46.936825 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f2270d-e827-4576-914c-9cd22286e017 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:46.972148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71e30b-5bd3-4a92-91cf-a1729d447f7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:46.972832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71e30b-5bd3-4a92-91cf-a1729d447f7c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:46.973130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71e30b-5bd3-4a92-91cf-a1729d447f7c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:46.973412 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc71e30b-5bd3-4a92-91cf-a1729d447f7c None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:46.974218 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71e30b-5bd3-4a92-91cf-a1729d447f7c None admin] RBAC: Adding query filter params (type=image) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:46.974562 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71e30b-5bd3-4a92-91cf-a1729d447f7c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:46.975971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc71e30b-5bd3-4a92-91cf-a1729d447f7c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:46.985714 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 520/1043] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:46 2022] GET /identity/v3/services?type=image => generated 350 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:46.986725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd2af7-1509-409c-acd0-9e2cbfb59fe8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:47.098095 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fabd2af7-1509-409c-acd0-9e2cbfb59fe8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:47.133263 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8c90fbe-4420-405f-891a-cf34d057f906 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:47.133841 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c90fbe-4420-405f-891a-cf34d057f906 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:47.134125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c90fbe-4420-405f-891a-cf34d057f906 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:47.134438 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8c90fbe-4420-405f-891a-cf34d057f906 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:47.135233 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c90fbe-4420-405f-891a-cf34d057f906 None admin] RBAC: Adding query filter params (interface=public, service_id=5daad888e1024c82ab3961717a1d8898, region_id=RegionOne) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:47.135602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c90fbe-4420-405f-891a-cf34d057f906 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:47.136957 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8c90fbe-4420-405f-891a-cf34d057f906 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:47.146632 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 524/1044] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Jul 11 13:54:46 2022] GET /identity/v3/endpoints?service_id=5daad888e1024c82ab3961717a1d8898&interface=public®ion_id=RegionOne => generated 194 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:47.148216 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71e30b-5bd3-4a92-91cf-a1729d447f7c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:47.256424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc71e30b-5bd3-4a92-91cf-a1729d447f7c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:47.266799 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:47.296501 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2918f947-f5aa-419a-9162-8184649abd7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:47.297287 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918f947-f5aa-419a-9162-8184649abd7f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:47.297581 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918f947-f5aa-419a-9162-8184649abd7f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:47.297862 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2918f947-f5aa-419a-9162-8184649abd7f None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:47.298588 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918f947-f5aa-419a-9162-8184649abd7f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:47.299984 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2918f947-f5aa-419a-9162-8184649abd7f None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:47.307439 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 521/1045] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Jul 11 13:54:47 2022] GET /identity/v3/services => generated 1957 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 13:54:47.509524 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cba4c-36f0-462c-9b48-462bfa86c72a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:47.509898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cba4c-36f0-462c-9b48-462bfa86c72a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:47.510096 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663cba4c-36f0-462c-9b48-462bfa86c72a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:47.511645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 525/1046] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:47.517415 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ab70b-3ce7-4472-93ad-2fe0eca233af None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:47.517769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ab70b-3ce7-4472-93ad-2fe0eca233af None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:47.518117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839ab70b-3ce7-4472-93ad-2fe0eca233af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:47.534818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-839ab70b-3ce7-4472-93ad-2fe0eca233af None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:47.536038 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-839ab70b-3ce7-4472-93ad-2fe0eca233af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:47.613193 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 522/1047] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:47 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:54:47.636590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cba4c-36f0-462c-9b48-462bfa86c72a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:47.666034 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:47.746115 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663cba4c-36f0-462c-9b48-462bfa86c72a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:47.782967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a427bb-d65d-4c51-b59f-b6fe9ec2c40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:47.783678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a427bb-d65d-4c51-b59f-b6fe9ec2c40f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:47.783955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a427bb-d65d-4c51-b59f-b6fe9ec2c40f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:47.784227 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a427bb-d65d-4c51-b59f-b6fe9ec2c40f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:47.795354 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a427bb-d65d-4c51-b59f-b6fe9ec2c40f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:54:47.802912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b2a427bb-d65d-4c51-b59f-b6fe9ec2c40f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:54:47.802912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:47.802912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:47.802912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:47.802912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:47.802912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:47.802912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:47.802912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:47.802912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:47.802912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:47.802912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:47.802912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:47.802912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:54:47.802912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:54:47.802912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:54:47.802912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:54:47.803867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:54:47.803867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:54:47.803867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:54:47.803867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:54:47.803867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:47.803867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:47.803867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:47.803867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:47.803867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:47.803867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:47.803867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:47.803867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:47.803867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:47.803867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:47.803867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:47.803867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:47.804952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:47.804952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:47.804952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:47.804952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:47.804952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:54:47.804952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:54:47.804952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:54:47.804952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:54:47.804952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:54:47.804952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:54:47.804952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:54:47.804952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:47.806387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 526/1048] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:54:47 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:47.810899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ab70b-3ce7-4472-93ad-2fe0eca233af None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:47.917225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839ab70b-3ce7-4472-93ad-2fe0eca233af None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:47.919677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:47.954066 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244e261-eaaf-407f-ac53-f218cdbcd393 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:47.954764 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244e261-eaaf-407f-ac53-f218cdbcd393 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:47.955034 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244e261-eaaf-407f-ac53-f218cdbcd393 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:47.955307 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2244e261-eaaf-407f-ac53-f218cdbcd393 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:47.956069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244e261-eaaf-407f-ac53-f218cdbcd393 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:47.956379 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244e261-eaaf-407f-ac53-f218cdbcd393 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:47.957740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2244e261-eaaf-407f-ac53-f218cdbcd393 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:47.971798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 523/1049] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:54:47 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:47.973061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a427bb-d65d-4c51-b59f-b6fe9ec2c40f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:48.063355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:48.081480 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a427bb-d65d-4c51-b59f-b6fe9ec2c40f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:48.117510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1da3f0b0-c6a9-48cb-8576-93d82f4c40c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:48.118409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3f0b0-c6a9-48cb-8576-93d82f4c40c5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:48.118735 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3f0b0-c6a9-48cb-8576-93d82f4c40c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:48.119014 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1da3f0b0-c6a9-48cb-8576-93d82f4c40c5 None admin] PATH_INFO: `/v3/users/b438c1d1d57f47ddbab2ec188d032872` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:48.132745 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3f0b0-c6a9-48cb-8576-93d82f4c40c5 None admin] RBAC: Authorizing `identity:get_user(user_id=b438c1d1d57f47ddbab2ec188d032872)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:48.134177 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1da3f0b0-c6a9-48cb-8576-93d82f4c40c5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:48.140386 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 527/1050] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:54:47 2022] GET /identity/v3/users/b438c1d1d57f47ddbab2ec188d032872 => generated 285 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:48.141634 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244e261-eaaf-407f-ac53-f218cdbcd393 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:48.244954 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2244e261-eaaf-407f-ac53-f218cdbcd393 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:48.279358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175e75d-6438-4839-99d4-b8b4390b2b56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:48.280039 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175e75d-6438-4839-99d4-b8b4390b2b56 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:48.280325 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175e75d-6438-4839-99d4-b8b4390b2b56 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:48.280601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6175e75d-6438-4839-99d4-b8b4390b2b56 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:48.297363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe25c6-f3ab-405b-9ea1-7926c7c0a30f None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:48.297654 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe25c6-f3ab-405b-9ea1-7926c7c0a30f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:48.297943 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfe25c6-f3ab-405b-9ea1-7926c7c0a30f None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:48.299597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 528/1051] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:54:48.304478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937bf32-fc93-4087-9ed6-1870272d9b38 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:48.305028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937bf32-fc93-4087-9ed6-1870272d9b38 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:48.305311 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e937bf32-fc93-4087-9ed6-1870272d9b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:48.317520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:48.322863 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e937bf32-fc93-4087-9ed6-1870272d9b38 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:48.324066 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e937bf32-fc93-4087-9ed6-1870272d9b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:48.359821 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175e75d-6438-4839-99d4-b8b4390b2b56 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:48.361289 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6175e75d-6438-4839-99d4-b8b4390b2b56 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:48.397202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 529/1052] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:48 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:54:48.420167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937bf32-fc93-4087-9ed6-1870272d9b38 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:48.476536 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 524/1053] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:54:48 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 13:54:48.477644 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175e75d-6438-4839-99d4-b8b4390b2b56 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:48.513178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:48.540513 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e937bf32-fc93-4087-9ed6-1870272d9b38 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:48.545265 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:48.579952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c08de-03bc-490e-9536-7464602ed29f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:48.580725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c08de-03bc-490e-9536-7464602ed29f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:48.580960 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c08de-03bc-490e-9536-7464602ed29f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:48.581228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c08de-03bc-490e-9536-7464602ed29f None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:48.588202 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6175e75d-6438-4839-99d4-b8b4390b2b56 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:48.591719 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f12c08de-03bc-490e-9536-7464602ed29f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:54:48.599962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-f12c08de-03bc-490e-9536-7464602ed29f None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 13:54:48.599962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:48.599962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:48.599962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:48.599962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:48.599962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:48.599962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:48.599962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:48.599962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:48.599962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:48.599962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:48.599962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:48.599962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:54:48.599962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:54:48.599962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:54:48.599962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:54:48.600888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:54:48.600888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:54:48.600888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:54:48.600888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:54:48.600888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:48.600888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:48.600888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:48.600888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:48.600888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:48.600888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:48.600888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:48.600888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:48.600888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:48.600888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:48.600888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:48.600888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:48.602662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:48.602662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:48.602662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:48.602662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:48.602662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:54:48.602662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:54:48.602662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:54:48.602662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:54:48.602662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:54:48.602662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:54:48.602662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 13:54:48.602662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:48.602662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 530/1054] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Jul 11 13:54:48 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:48.607576 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c08de-03bc-490e-9536-7464602ed29f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:48.628480 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec126f2-00d1-4672-ab47-6baed92d948b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:48.629345 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec126f2-00d1-4672-ab47-6baed92d948b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:48.629631 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec126f2-00d1-4672-ab47-6baed92d948b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:48.629909 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec126f2-00d1-4672-ab47-6baed92d948b None admin] PATH_INFO: `/v3/projects/7eb1e22817e14f85accda9fa9c9aa6e6` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:48.631130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec126f2-00d1-4672-ab47-6baed92d948b None admin] RBAC: Authorizing `identity:get_project(project_id=7eb1e22817e14f85accda9fa9c9aa6e6)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:48.632563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ec126f2-00d1-4672-ab47-6baed92d948b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:48.639280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 525/1055] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:54:48 2022] GET /identity/v3/projects/7eb1e22817e14f85accda9fa9c9aa6e6 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:48.640745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec126f2-00d1-4672-ab47-6baed92d948b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:48.713530 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c08de-03bc-490e-9536-7464602ed29f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:48.747130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec126f2-00d1-4672-ab47-6baed92d948b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:48.771076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6148a-de21-4cf9-8ff2-55addf1a85bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:48.771842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6148a-de21-4cf9-8ff2-55addf1a85bd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:48.772132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6148a-de21-4cf9-8ff2-55addf1a85bd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:48.772413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f6148a-de21-4cf9-8ff2-55addf1a85bd None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:48.773298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6148a-de21-4cf9-8ff2-55addf1a85bd None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:48.773619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6148a-de21-4cf9-8ff2-55addf1a85bd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:48.775217 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9f6148a-de21-4cf9-8ff2-55addf1a85bd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:48.777904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:48.787305 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 531/1056] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Jul 11 13:54:48 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:48.796006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:48.796313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6148a-de21-4cf9-8ff2-55addf1a85bd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:48.801942 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd3c91-973f-4759-96b2-3761cb5ed381 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:48.802648 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd3c91-973f-4759-96b2-3761cb5ed381 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:48.802940 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd3c91-973f-4759-96b2-3761cb5ed381 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:48.803223 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fd3c91-973f-4759-96b2-3761cb5ed381 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:48.804148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd3c91-973f-4759-96b2-3761cb5ed381 None admin] RBAC: Adding query filter params (role.id=6964639a7f6c41e8a6701ef533151c41, scope.project.id=7eb1e22817e14f85accda9fa9c9aa6e6, user.id=b438c1d1d57f47ddbab2ec188d032872) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:48.805118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd3c91-973f-4759-96b2-3761cb5ed381 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:48.806885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fd3c91-973f-4759-96b2-3761cb5ed381 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:48.813372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 526/1057] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:54:48 2022] GET /identity/v3/role_assignments?user.id=b438c1d1d57f47ddbab2ec188d032872&scope.project.id=7eb1e22817e14f85accda9fa9c9aa6e6&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 259 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:48.928556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f6148a-de21-4cf9-8ff2-55addf1a85bd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:48.967335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab9745-3d7e-4a2b-922b-549f3517b7d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:48.968145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab9745-3d7e-4a2b-922b-549f3517b7d7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:48.968437 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab9745-3d7e-4a2b-922b-549f3517b7d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:48.968716 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ab9745-3d7e-4a2b-922b-549f3517b7d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:48.982610 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ab9745-3d7e-4a2b-922b-549f3517b7d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:48.983769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ab9745-3d7e-4a2b-922b-549f3517b7d7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:49.002259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-61ab9745-3d7e-4a2b-922b-549f3517b7d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:49.002259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:49.002259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:49.002259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:49.002259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:49.002259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:49.002259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:49.002259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:49.002259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:49.002259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:49.002259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:49.002259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:49.002259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:49.002259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:49.002259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:49.003318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:49.003318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:49.003318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:49.003318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:49.003318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:49.003318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:49.003318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:49.003318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:49.003318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:49.003318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:49.003318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:49.003318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:49.003318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:49.003318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:49.003318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:49.003318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:49.003318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:49.004202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:49.004202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:49.004202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:49.004202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:49.004202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 532/1058] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:49.009206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd3c91-973f-4759-96b2-3761cb5ed381 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:49.124775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fd3c91-973f-4759-96b2-3761cb5ed381 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:49.159920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef127fa-c2e6-4824-bbab-1c426d197a2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:49.160585 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef127fa-c2e6-4824-bbab-1c426d197a2b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:49.160869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef127fa-c2e6-4824-bbab-1c426d197a2b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:49.161140 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef127fa-c2e6-4824-bbab-1c426d197a2b None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:49.161863 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef127fa-c2e6-4824-bbab-1c426d197a2b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:49.162167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef127fa-c2e6-4824-bbab-1c426d197a2b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:49.163594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef127fa-c2e6-4824-bbab-1c426d197a2b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:49.181137 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 527/1059] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:49.182840 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab9745-3d7e-4a2b-922b-549f3517b7d7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:49.235836 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:49.288137 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ab9745-3d7e-4a2b-922b-549f3517b7d7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:49.322278 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e50d4b5a-bdc2-4ead-9e4b-73e6290ca479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:49.323102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d4b5a-bdc2-4ead-9e4b-73e6290ca479 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:49.323384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d4b5a-bdc2-4ead-9e4b-73e6290ca479 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:49.323659 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e50d4b5a-bdc2-4ead-9e4b-73e6290ca479 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:49.331652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d4b5a-bdc2-4ead-9e4b-73e6290ca479 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:49.333085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e50d4b5a-bdc2-4ead-9e4b-73e6290ca479 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:49.349015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-e50d4b5a-bdc2-4ead-9e4b-73e6290ca479 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 13:54:49.349015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:49.349015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:49.349015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:49.349015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:49.349015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:49.349015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:49.349015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:49.349015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:49.349015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:49.349015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:49.349015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:49.349015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:54:49.349015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:54:49.349015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:54:49.349015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:54:49.349985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:49.349985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:49.349985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:54:49.349985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:49.349985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:54:49.349985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:49.349985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:49.349985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:49.349985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:49.349985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:49.349985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:49.349985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:49.349985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:49.349985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:49.349985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:49.349985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:49.349985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:49.349985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:49.351009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:49.351009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:49.351009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:54:49.351009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:54:49.351009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:54:49.351009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:54:49.351009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:54:49.351009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:54:49.351009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 13:54:49.351009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:49.354630 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 533/1060] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Jul 11 13:54:49 2022] GET /identity/v3/users/nova => generated 82 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:49.356063 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef127fa-c2e6-4824-bbab-1c426d197a2b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:49.373919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc91cc4-67be-4690-a593-79fe2b70d81e None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:49.374325 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc91cc4-67be-4690-a593-79fe2b70d81e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:49.374630 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cc91cc4-67be-4690-a593-79fe2b70d81e None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:49.376134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 534/1061] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:54:49.381081 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4333225-d6b2-41f7-87b9-4608892d70dc None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:49.381361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4333225-d6b2-41f7-87b9-4608892d70dc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:49.381694 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4333225-d6b2-41f7-87b9-4608892d70dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:49.398924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b4333225-d6b2-41f7-87b9-4608892d70dc None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:49.400112 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b4333225-d6b2-41f7-87b9-4608892d70dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:49.421939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:49.472536 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 535/1062] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:49 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:54:49.474905 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef127fa-c2e6-4824-bbab-1c426d197a2b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:49.495270 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4333225-d6b2-41f7-87b9-4608892d70dc None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:49.510453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9d2c2-0a9f-4f56-8947-ed2c82d27725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:49.511127 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9d2c2-0a9f-4f56-8947-ed2c82d27725 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:49.511409 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9d2c2-0a9f-4f56-8947-ed2c82d27725 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:49.511686 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9d2c2-0a9f-4f56-8947-ed2c82d27725 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:49.512585 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9d2c2-0a9f-4f56-8947-ed2c82d27725 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:49.513390 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9d2c2-0a9f-4f56-8947-ed2c82d27725 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:49.514923 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31e9d2c2-0a9f-4f56-8947-ed2c82d27725 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:49.534338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 528/1063] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Jul 11 13:54:49 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:49.538502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9d2c2-0a9f-4f56-8947-ed2c82d27725 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:49.542907 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:49.610846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4333225-d6b2-41f7-87b9-4608892d70dc None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:49.645489 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9d2c2-0a9f-4f56-8947-ed2c82d27725 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:49.648149 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed38224-d941-4719-b6c9-3e21930dee28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:49.648917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed38224-d941-4719-b6c9-3e21930dee28 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:49.649204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed38224-d941-4719-b6c9-3e21930dee28 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:49.649482 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ed38224-d941-4719-b6c9-3e21930dee28 None admin] PATH_INFO: `/v3/services/image` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:49.650147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed38224-d941-4719-b6c9-3e21930dee28 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:49.651554 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ed38224-d941-4719-b6c9-3e21930dee28 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:49.663322 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:49.663900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-0ed38224-d941-4719-b6c9-3e21930dee28 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 13:54:49.663900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:49.663900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:54:49.663900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:54:49.663900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:54:49.663900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:54:49.663900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:54:49.663900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:49.663900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 13:54:49.663900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:49.663900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:54:49.663900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:49.663900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:49.663900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:49.663900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:49.663900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:49.663900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:49.664905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:49.664905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:49.664905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:49.664905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:49.664905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:49.664905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:49.664905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:54:49.664905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:54:49.664905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:54:49.664905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:54:49.664905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:49.664905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:49.664905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:49.664905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:49.664905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:49.664905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:49.665728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:49.665728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:49.665728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:49.665728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:49.665728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:49.665728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:49.665728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:49.665728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:49.665728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:49.665728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:49.665728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:54:49.665728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:54:49.665728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 11 13:54:49.665728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:49.665728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 536/1064] 10.222.0.49 () {56 vars in 1105 bytes} [Mon Jul 11 13:54:49 2022] GET /identity/v3/services/image => generated 86 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:49.672481 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed38224-d941-4719-b6c9-3e21930dee28 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:49.693236 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1f7d7-a360-48bd-9bba-3508373ad1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:49.694583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1f7d7-a360-48bd-9bba-3508373ad1a8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:49.695093 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1f7d7-a360-48bd-9bba-3508373ad1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:49.696088 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b1f7d7-a360-48bd-9bba-3508373ad1a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:49.710646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1f7d7-a360-48bd-9bba-3508373ad1a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:49.712331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b1f7d7-a360-48bd-9bba-3508373ad1a8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:49.736117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-c6b1f7d7-a360-48bd-9bba-3508373ad1a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:49.736117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:49.736117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:49.736117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:49.736117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:49.736117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:49.736117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:49.736117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:49.736117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:49.736117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:49.736117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:49.736117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:49.736117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:49.736117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:49.736117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:49.737018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:49.737018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:49.737018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:49.737018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:49.737018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:49.737018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:49.737018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:49.737018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:49.737018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:49.737018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:49.737018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:49.737018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:49.737018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:49.737018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:49.737018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:49.737018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:49.737018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:49.737903 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:49.737903 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:49.737903 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:49.737903 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:49.738136 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 529/1065] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:49.743857 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1f7d7-a360-48bd-9bba-3508373ad1a8 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:49.781116 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ed38224-d941-4719-b6c9-3e21930dee28 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:49.784359 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:49.819994 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653cb8c8-dd41-4eee-acd8-22cfc3850618 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:49.820671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653cb8c8-dd41-4eee-acd8-22cfc3850618 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:49.820958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653cb8c8-dd41-4eee-acd8-22cfc3850618 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:49.821238 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653cb8c8-dd41-4eee-acd8-22cfc3850618 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:49.821954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653cb8c8-dd41-4eee-acd8-22cfc3850618 None admin] RBAC: Adding query filter params (name=image) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:49.822266 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653cb8c8-dd41-4eee-acd8-22cfc3850618 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:49.823719 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653cb8c8-dd41-4eee-acd8-22cfc3850618 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:49.829364 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 537/1066] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:49 2022] GET /identity/v3/services?name=image => generated 122 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:49.834964 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653cb8c8-dd41-4eee-acd8-22cfc3850618 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:49.857534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b1f7d7-a360-48bd-9bba-3508373ad1a8 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:49.893754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb52dc-ef78-428d-aa8b-c57076ca5db8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:49.894468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb52dc-ef78-428d-aa8b-c57076ca5db8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:49.894734 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb52dc-ef78-428d-aa8b-c57076ca5db8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:49.895011 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bdb52dc-ef78-428d-aa8b-c57076ca5db8 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:49.895747 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb52dc-ef78-428d-aa8b-c57076ca5db8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:49.896052 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb52dc-ef78-428d-aa8b-c57076ca5db8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:49.897431 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bdb52dc-ef78-428d-aa8b-c57076ca5db8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:49.910625 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 530/1067] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:49.916138 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb52dc-ef78-428d-aa8b-c57076ca5db8 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:49.950775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:49.957770 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653cb8c8-dd41-4eee-acd8-22cfc3850618 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:49.997481 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3d19f-7fba-4b7d-9566-14cec0b4f685 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:49.998110 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3d19f-7fba-4b7d-9566-14cec0b4f685 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:49.998801 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3d19f-7fba-4b7d-9566-14cec0b4f685 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:49.998801 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c3d19f-7fba-4b7d-9566-14cec0b4f685 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:49.999334 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c3d19f-7fba-4b7d-9566-14cec0b4f685 None admin] RBAC: Adding query filter params (type=image) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:49.999637 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c3d19f-7fba-4b7d-9566-14cec0b4f685 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:50.001019 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88c3d19f-7fba-4b7d-9566-14cec0b4f685 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:50.011653 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 538/1068] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:49 2022] GET /identity/v3/services?type=image => generated 350 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:50.014426 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3d19f-7fba-4b7d-9566-14cec0b4f685 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:50.041936 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bdb52dc-ef78-428d-aa8b-c57076ca5db8 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:50.049827 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:50.094887 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba6977-eb78-42f9-99cb-78e6dc2b9b7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:50.095683 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba6977-eb78-42f9-99cb-78e6dc2b9b7e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:50.095966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba6977-eb78-42f9-99cb-78e6dc2b9b7e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:50.096243 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bba6977-eb78-42f9-99cb-78e6dc2b9b7e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:50.103372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba6977-eb78-42f9-99cb-78e6dc2b9b7e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:50.104823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bba6977-eb78-42f9-99cb-78e6dc2b9b7e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:50.119738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-1bba6977-eb78-42f9-99cb-78e6dc2b9b7e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:50.119738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:50.119738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:50.119738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:50.119738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:50.119738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:50.119738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:50.119738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:50.119738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:50.119738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:50.119738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:50.119738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:50.119738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:54:50.119738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:54:50.119738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:54:50.120696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:54:50.120696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:50.120696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:50.120696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:50.120696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:50.120696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:50.120696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:50.120696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:50.120696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:50.120696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:50.120696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:50.120696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:50.120696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:50.120696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:50.120696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:50.120696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:50.120696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:50.121589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:54:50.121589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:54:50.121589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:54:50.121589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:54:50.121589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:54:50.121589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:54:50.121589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:50.121589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:50.121589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 531/1069] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:49 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:50.126783 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba6977-eb78-42f9-99cb-78e6dc2b9b7e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:50.131475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c3d19f-7fba-4b7d-9566-14cec0b4f685 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:50.168372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e104e9-9f32-461e-9cfe-d6a622c86f80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:50.169067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e104e9-9f32-461e-9cfe-d6a622c86f80 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:50.169651 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e104e9-9f32-461e-9cfe-d6a622c86f80 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:50.169651 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e104e9-9f32-461e-9cfe-d6a622c86f80 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:50.170385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e104e9-9f32-461e-9cfe-d6a622c86f80 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:50.171741 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e104e9-9f32-461e-9cfe-d6a622c86f80 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:50.193093 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 539/1070] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Jul 11 13:54:50 2022] POST /identity/v3/endpoints => generated 327 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 13:54:50.224021 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:50.244401 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bba6977-eb78-42f9-99cb-78e6dc2b9b7e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:50.292077 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b42218-379c-4734-b54a-05d85bf791f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:50.292664 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b42218-379c-4734-b54a-05d85bf791f0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:50.292941 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b42218-379c-4734-b54a-05d85bf791f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:50.293210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b42218-379c-4734-b54a-05d85bf791f0 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:50.294054 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b42218-379c-4734-b54a-05d85bf791f0 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:50.294454 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b42218-379c-4734-b54a-05d85bf791f0 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:50.296051 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b42218-379c-4734-b54a-05d85bf791f0 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:50.315559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 532/1071] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:54:50 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:50.316831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e104e9-9f32-461e-9cfe-d6a622c86f80 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:50.418760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:50.435141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e104e9-9f32-461e-9cfe-d6a622c86f80 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:50.471310 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa2697e-d7b4-4d34-9a06-5df8b387c58b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:50.472214 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2697e-d7b4-4d34-9a06-5df8b387c58b None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:50.472503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2697e-d7b4-4d34-9a06-5df8b387c58b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:50.472783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa2697e-d7b4-4d34-9a06-5df8b387c58b None admin] PATH_INFO: `/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/ef5dac3f311e43809a46995bab56e780/roles/836753c7d9b44fde817a38c1566d0659` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:50.475097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2697e-d7b4-4d34-9a06-5df8b387c58b None admin] RBAC: Authorizing `identity:create_grant(project_id=08ee65f8b338491f80e7beac5e316b23, user_id=ef5dac3f311e43809a46995bab56e780, role_id=836753c7d9b44fde817a38c1566d0659)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:50.476522 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa2697e-d7b4-4d34-9a06-5df8b387c58b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:50.497384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 540/1072] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:54:50 2022] PUT /identity/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/ef5dac3f311e43809a46995bab56e780/roles/836753c7d9b44fde817a38c1566d0659 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 13:54:50.922206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367c6a0-b74b-462e-833d-7df5b13beb1d None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:50.922619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367c6a0-b74b-462e-833d-7df5b13beb1d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:50.922818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3367c6a0-b74b-462e-833d-7df5b13beb1d None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:50.924169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 533/1073] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:50.930228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda55a3a-b968-4f7c-9a57-ce94f921f977 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:50.930554 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda55a3a-b968-4f7c-9a57-ce94f921f977 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:50.930834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda55a3a-b968-4f7c-9a57-ce94f921f977 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:50.947818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cda55a3a-b968-4f7c-9a57-ce94f921f977 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:50.949005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cda55a3a-b968-4f7c-9a57-ce94f921f977 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:51.019847 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 541/1074] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:50 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:51.042394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367c6a0-b74b-462e-833d-7df5b13beb1d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:51.191047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3367c6a0-b74b-462e-833d-7df5b13beb1d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:51.229989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8569f-264a-4e7a-ad94-b02e8a2ebf49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:51.230813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8569f-264a-4e7a-ad94-b02e8a2ebf49 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:51.231094 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8569f-264a-4e7a-ad94-b02e8a2ebf49 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:51.231939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4be8569f-264a-4e7a-ad94-b02e8a2ebf49 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:51.242962 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4be8569f-264a-4e7a-ad94-b02e8a2ebf49 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:54:51.250860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-4be8569f-264a-4e7a-ad94-b02e8a2ebf49 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:54:51.250860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:51.250860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:51.250860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:51.250860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:51.250860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:51.250860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:51.250860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:51.250860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:51.250860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:51.250860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:51.250860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:51.250860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:54:51.250860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:54:51.250860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:54:51.250860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:54:51.251809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:54:51.251809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:54:51.251809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:54:51.251809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:54:51.251809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:51.251809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:51.251809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:51.251809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:51.251809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:51.251809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:51.251809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:51.251809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:51.251809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:51.251809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:51.251809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:51.251809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:51.252651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:51.252651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:51.252651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:51.252651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:51.252651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:54:51.252651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:54:51.252651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:54:51.252651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:54:51.252651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:54:51.252651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:54:51.252651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:54:51.252651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:51.254367 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 534/1075] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:54:51 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:51.258492 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda55a3a-b968-4f7c-9a57-ce94f921f977 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:51.341774 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:51.368632 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda55a3a-b968-4f7c-9a57-ce94f921f977 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:51.404802 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68370e-3e93-4466-a27b-f3eaf3c9f78d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:51.405472 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68370e-3e93-4466-a27b-f3eaf3c9f78d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:51.405757 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68370e-3e93-4466-a27b-f3eaf3c9f78d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:51.406031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c68370e-3e93-4466-a27b-f3eaf3c9f78d None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:51.406812 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68370e-3e93-4466-a27b-f3eaf3c9f78d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:51.407121 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68370e-3e93-4466-a27b-f3eaf3c9f78d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:51.408477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c68370e-3e93-4466-a27b-f3eaf3c9f78d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:51.421432 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 542/1076] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:54:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:51.423196 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8569f-264a-4e7a-ad94-b02e8a2ebf49 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:51.426007 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:51.526822 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4be8569f-264a-4e7a-ad94-b02e8a2ebf49 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:51.562816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34729b82-ae9d-4114-891f-16660c3e16c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:51.563720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34729b82-ae9d-4114-891f-16660c3e16c8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:51.563999 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34729b82-ae9d-4114-891f-16660c3e16c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:51.564286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34729b82-ae9d-4114-891f-16660c3e16c8 None admin] PATH_INFO: `/v3/users/b438c1d1d57f47ddbab2ec188d032872` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:51.565472 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34729b82-ae9d-4114-891f-16660c3e16c8 None admin] RBAC: Authorizing `identity:get_user(user_id=b438c1d1d57f47ddbab2ec188d032872)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:51.567118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34729b82-ae9d-4114-891f-16660c3e16c8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:51.573960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 535/1077] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:54:51 2022] GET /identity/v3/users/b438c1d1d57f47ddbab2ec188d032872 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:51.576272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68370e-3e93-4466-a27b-f3eaf3c9f78d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:51.691065 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c68370e-3e93-4466-a27b-f3eaf3c9f78d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:51.725994 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c172deb-472a-4fd8-a6cb-ec8f4abf92d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:51.726899 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c172deb-472a-4fd8-a6cb-ec8f4abf92d0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:51.727185 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c172deb-472a-4fd8-a6cb-ec8f4abf92d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:51.727464 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c172deb-472a-4fd8-a6cb-ec8f4abf92d0 None admin] PATH_INFO: `/v3/projects/7eb1e22817e14f85accda9fa9c9aa6e6` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:51.728654 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c172deb-472a-4fd8-a6cb-ec8f4abf92d0 None admin] RBAC: Authorizing `identity:get_project(project_id=7eb1e22817e14f85accda9fa9c9aa6e6)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:51.730083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c172deb-472a-4fd8-a6cb-ec8f4abf92d0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:51.737313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 543/1078] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:54:51 2022] GET /identity/v3/projects/7eb1e22817e14f85accda9fa9c9aa6e6 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:51.739578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34729b82-ae9d-4114-891f-16660c3e16c8 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:51.812074 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:51.844729 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34729b82-ae9d-4114-891f-16660c3e16c8 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:51.880193 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f2218c-636c-4e54-8eb8-af61c2e870ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:51.881087 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2218c-636c-4e54-8eb8-af61c2e870ae None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:51.881371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2218c-636c-4e54-8eb8-af61c2e870ae None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:51.881651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f2218c-636c-4e54-8eb8-af61c2e870ae None admin] PATH_INFO: `/v3/projects/7eb1e22817e14f85accda9fa9c9aa6e6/users/b438c1d1d57f47ddbab2ec188d032872/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:51.883641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f2218c-636c-4e54-8eb8-af61c2e870ae None admin] RBAC: Authorizing `identity:create_grant(project_id=7eb1e22817e14f85accda9fa9c9aa6e6, user_id=b438c1d1d57f47ddbab2ec188d032872, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:51.885081 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f2218c-636c-4e54-8eb8-af61c2e870ae None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:51.899905 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 536/1079] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:54:51 2022] PUT /identity/v3/projects/7eb1e22817e14f85accda9fa9c9aa6e6/users/b438c1d1d57f47ddbab2ec188d032872/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:54:52.321905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d063230-9034-4753-86bb-8e32f4c39a28 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:52.322559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d063230-9034-4753-86bb-8e32f4c39a28 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:52.322559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d063230-9034-4753-86bb-8e32f4c39a28 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:52.323892 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 544/1080] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:52.329883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a3812-80a5-4bcb-97a3-e3ff928f9145 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:52.330662 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a3812-80a5-4bcb-97a3-e3ff928f9145 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:52.330662 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733a3812-80a5-4bcb-97a3-e3ff928f9145 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:52.348738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-733a3812-80a5-4bcb-97a3-e3ff928f9145 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:52.349945 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-733a3812-80a5-4bcb-97a3-e3ff928f9145 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:52.425376 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 537/1081] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:52.448583 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d063230-9034-4753-86bb-8e32f4c39a28 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:52.462941 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:52.560916 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d063230-9034-4753-86bb-8e32f4c39a28 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:52.596910 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3750448-da26-451e-84b0-a7bf2afbcb0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:52.597688 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3750448-da26-451e-84b0-a7bf2afbcb0d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:52.597980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3750448-da26-451e-84b0-a7bf2afbcb0d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:52.598261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3750448-da26-451e-84b0-a7bf2afbcb0d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:52.603364 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420cc61-7137-4422-aa79-7b176f9c1103 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:52.603661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420cc61-7137-4422-aa79-7b176f9c1103 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:52.603932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2420cc61-7137-4422-aa79-7b176f9c1103 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:52.605456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 538/1082] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:52.611126 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d71d30-ae52-4b56-af0f-541678a3ea65 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:52.611424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d71d30-ae52-4b56-af0f-541678a3ea65 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:52.611704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d71d30-ae52-4b56-af0f-541678a3ea65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:52.628393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b9d71d30-ae52-4b56-af0f-541678a3ea65 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:52.629604 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b9d71d30-ae52-4b56-af0f-541678a3ea65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:52.670968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3750448-da26-451e-84b0-a7bf2afbcb0d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:52.672420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3750448-da26-451e-84b0-a7bf2afbcb0d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:52.684854 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:52.696461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:52.710859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 539/1083] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:52.737863 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d71d30-ae52-4b56-af0f-541678a3ea65 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:52.790578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 545/1084] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:54:52 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 13:54:52.796975 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3750448-da26-451e-84b0-a7bf2afbcb0d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:52.847884 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d71d30-ae52-4b56-af0f-541678a3ea65 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:52.883328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1213a0-f2cd-463f-ae60-c9598243b736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:52.884160 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1213a0-f2cd-463f-ae60-c9598243b736 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:52.884449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1213a0-f2cd-463f-ae60-c9598243b736 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:52.884726 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f1213a0-f2cd-463f-ae60-c9598243b736 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:52.896059 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f1213a0-f2cd-463f-ae60-c9598243b736 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:54:52.903927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3750448-da26-451e-84b0-a7bf2afbcb0d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:52.904897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-7f1213a0-f2cd-463f-ae60-c9598243b736 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 13:54:52.904897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:52.904897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:52.904897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:52.904897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:52.904897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:52.904897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:52.904897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:52.904897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:52.904897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:52.904897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:52.904897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:52.904897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:54:52.904897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:54:52.904897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:54:52.904897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:54:52.905885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:54:52.905885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:54:52.905885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:54:52.905885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:54:52.905885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:52.905885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:52.905885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:52.905885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:52.905885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:52.905885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:52.905885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:52.905885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:52.905885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:52.905885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:52.905885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:52.905885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:52.906837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:52.906837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:52.906837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:52.906837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:52.906837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:54:52.906837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:54:52.906837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:54:52.906837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:54:52.906837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:54:52.906837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:54:52.906837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 11 13:54:52.906837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:52.911096 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 540/1085] 10.222.0.49 () {56 vars in 1120 bytes} [Mon Jul 11 13:54:52 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:52.911875 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1213a0-f2cd-463f-ae60-c9598243b736 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:52.917294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:52.947461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834cc12-7316-4716-92ae-1d512e1982bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:52.948523 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834cc12-7316-4716-92ae-1d512e1982bc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:52.948981 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834cc12-7316-4716-92ae-1d512e1982bc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:52.949259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c834cc12-7316-4716-92ae-1d512e1982bc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:52.961073 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834cc12-7316-4716-92ae-1d512e1982bc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:52.962719 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c834cc12-7316-4716-92ae-1d512e1982bc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:52.967475 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:52.983811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c834cc12-7316-4716-92ae-1d512e1982bc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:52.983811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:52.983811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:52.983811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:52.983811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:52.983811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:52.983811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:52.983811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:52.983811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:52.983811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:52.983811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:52.983811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:52.983811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:52.983811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:52.983811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:52.985031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:52.985031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:52.985031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:52.985031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:52.985031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:52.985031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:52.985031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:52.985031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:52.985031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:52.985031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:52.985031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:52.985031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:52.985031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:52.985031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:52.985031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:52.985031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:52.985031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:52.985941 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:52.985941 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:52.985941 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:52.985941 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:52.985941 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 546/1086] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:52.991203 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834cc12-7316-4716-92ae-1d512e1982bc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:53.029005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f1213a0-f2cd-463f-ae60-c9598243b736 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:53.072731 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15515ad3-37a7-4d72-931e-ac9c37e967d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:53.073574 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15515ad3-37a7-4d72-931e-ac9c37e967d6 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:53.073862 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15515ad3-37a7-4d72-931e-ac9c37e967d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:53.074141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15515ad3-37a7-4d72-931e-ac9c37e967d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:53.078689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15515ad3-37a7-4d72-931e-ac9c37e967d6 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:53.078689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15515ad3-37a7-4d72-931e-ac9c37e967d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:53.078689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15515ad3-37a7-4d72-931e-ac9c37e967d6 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:53.086077 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 541/1087] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Jul 11 13:54:52 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:53.091527 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15515ad3-37a7-4d72-931e-ac9c37e967d6 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:53.120182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c834cc12-7316-4716-92ae-1d512e1982bc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:53.160174 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730c63b-1812-48a9-a389-840801232c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:53.160850 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730c63b-1812-48a9-a389-840801232c57 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:53.161145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730c63b-1812-48a9-a389-840801232c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:53.161414 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f730c63b-1812-48a9-a389-840801232c57 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:53.162134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730c63b-1812-48a9-a389-840801232c57 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:53.162481 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730c63b-1812-48a9-a389-840801232c57 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:53.163839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f730c63b-1812-48a9-a389-840801232c57 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:53.178347 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 547/1088] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:53.204112 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15515ad3-37a7-4d72-931e-ac9c37e967d6 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:53.219598 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:53.245168 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b93f4-e642-483d-93cd-6cd89079a66c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:53.245986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b93f4-e642-483d-93cd-6cd89079a66c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:53.246269 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b93f4-e642-483d-93cd-6cd89079a66c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:53.246578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12b93f4-e642-483d-93cd-6cd89079a66c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:53.258900 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b93f4-e642-483d-93cd-6cd89079a66c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:53.260347 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12b93f4-e642-483d-93cd-6cd89079a66c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:53.280861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-e12b93f4-e642-483d-93cd-6cd89079a66c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:53.280861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:53.280861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:53.280861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:53.280861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:53.280861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:53.280861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:53.280861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:53.280861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:53.280861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:53.280861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:53.280861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:53.280861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:53.280861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:53.280861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:53.281843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:53.281843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:53.281843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:53.281843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:53.281843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:53.281843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:53.281843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:53.281843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:53.281843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:53.281843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:53.281843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:53.281843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:53.281843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:53.281843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:53.281843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:53.281843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:53.281843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:53.283196 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:53.283196 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:53.283196 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:53.283196 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:53.286707 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 542/1089] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:53.287972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730c63b-1812-48a9-a389-840801232c57 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:53.314401 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:53.401215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f730c63b-1812-48a9-a389-840801232c57 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:53.440668 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d68d0-6070-48ec-bcf8-b717cf9d6c17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:53.441339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d68d0-6070-48ec-bcf8-b717cf9d6c17 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:53.441623 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d68d0-6070-48ec-bcf8-b717cf9d6c17 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:53.441898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de8d68d0-6070-48ec-bcf8-b717cf9d6c17 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:53.442813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d68d0-6070-48ec-bcf8-b717cf9d6c17 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:53.443121 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d68d0-6070-48ec-bcf8-b717cf9d6c17 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:53.444482 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de8d68d0-6070-48ec-bcf8-b717cf9d6c17 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:53.457685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 548/1090] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:53.463268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b93f4-e642-483d-93cd-6cd89079a66c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:53.580475 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12b93f4-e642-483d-93cd-6cd89079a66c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:53.614904 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f95356-18c9-4347-9ca9-42968e98a1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:53.615708 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f95356-18c9-4347-9ca9-42968e98a1bd None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:53.615993 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f95356-18c9-4347-9ca9-42968e98a1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:53.616271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f95356-18c9-4347-9ca9-42968e98a1bd None admin] PATH_INFO: `/v3/users/nova` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:53.624269 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f95356-18c9-4347-9ca9-42968e98a1bd None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:53.625716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f95356-18c9-4347-9ca9-42968e98a1bd None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:53.641246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-11f95356-18c9-4347-9ca9-42968e98a1bd None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 11 13:54:53.641246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:53.641246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:53.641246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:53.641246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:53.641246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:53.641246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:53.641246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:53.641246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:53.641246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:53.641246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:53.641246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:53.641246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:54:53.641246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:54:53.641246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:54:53.641246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:54:53.642541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:53.642541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:53.642541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:54:53.642541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:53.642541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:54:53.642541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:54:53.642541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:53.642541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:53.642541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:53.642541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:53.642541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:53.642541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:53.642541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:53.642541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:53.642541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:53.642541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:53.642541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:53.642541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:53.648163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:53.648163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:53.648163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:54:53.648163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:54:53.648163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:54:53.648163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:54:53.648163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:54:53.648163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:54:53.648163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 11 13:54:53.648163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:53.648163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 543/1091] 10.222.0.49 () {56 vars in 1093 bytes} [Mon Jul 11 13:54:53 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:53.648793 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d68d0-6070-48ec-bcf8-b717cf9d6c17 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:53.728779 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:53.747835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de8d68d0-6070-48ec-bcf8-b717cf9d6c17 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:53.785086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fd97d14-e19c-4e62-84d6-5321a43ea91e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:53.786415 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd97d14-e19c-4e62-84d6-5321a43ea91e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:53.787168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd97d14-e19c-4e62-84d6-5321a43ea91e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:53.787710 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fd97d14-e19c-4e62-84d6-5321a43ea91e None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:53.788694 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd97d14-e19c-4e62-84d6-5321a43ea91e None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:53.789059 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd97d14-e19c-4e62-84d6-5321a43ea91e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:53.790621 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fd97d14-e19c-4e62-84d6-5321a43ea91e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:53.809433 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 549/1092] 10.222.0.49 () {56 vars in 1133 bytes} [Mon Jul 11 13:54:53 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:53.815026 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f95356-18c9-4347-9ca9-42968e98a1bd None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:53.841457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:53.916862 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f95356-18c9-4347-9ca9-42968e98a1bd None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:53.950642 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4414306-408f-479b-903c-d3e1ef477889 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:53.951347 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4414306-408f-479b-903c-d3e1ef477889 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:53.951636 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4414306-408f-479b-903c-d3e1ef477889 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:53.951910 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4414306-408f-479b-903c-d3e1ef477889 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:54.017064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb78ecf-a9b1-43f6-a919-87783a907289 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:54.017376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb78ecf-a9b1-43f6-a919-87783a907289 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:54.017680 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb78ecf-a9b1-43f6-a919-87783a907289 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:54.019167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 550/1093] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:54:54.021664 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4414306-408f-479b-903c-d3e1ef477889 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:54.023154 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4414306-408f-479b-903c-d3e1ef477889 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:54.024120 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80870164-efe2-454c-87fa-5ce059737fdc None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:54.024404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80870164-efe2-454c-87fa-5ce059737fdc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:54.024683 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80870164-efe2-454c-87fa-5ce059737fdc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:54.041046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-80870164-efe2-454c-87fa-5ce059737fdc None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:54.042246 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-80870164-efe2-454c-87fa-5ce059737fdc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:54.049374 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:54.117671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 551/1094] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:54:54.135324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 544/1095] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:54:53 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 13:54:54.136412 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80870164-efe2-454c-87fa-5ce059737fdc None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:54.143271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4414306-408f-479b-903c-d3e1ef477889 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:54.204017 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:54.237365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80870164-efe2-454c-87fa-5ce059737fdc None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:54.243070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4414306-408f-479b-903c-d3e1ef477889 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:54.267886 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:54.272975 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4548f-616b-4edd-9339-fd3981386a75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:54.273768 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4548f-616b-4edd-9339-fd3981386a75 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:54.274054 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4548f-616b-4edd-9339-fd3981386a75 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:54.274344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc4548f-616b-4edd-9339-fd3981386a75 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:54.279138 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9ee8c-eddd-4eed-88cf-59fb72e28486 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:54.279929 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9ee8c-eddd-4eed-88cf-59fb72e28486 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:54.280213 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9ee8c-eddd-4eed-88cf-59fb72e28486 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:54.280489 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e9ee8c-eddd-4eed-88cf-59fb72e28486 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:54.284514 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc4548f-616b-4edd-9339-fd3981386a75 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:54.285956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc4548f-616b-4edd-9339-fd3981386a75 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:54.290581 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e9ee8c-eddd-4eed-88cf-59fb72e28486 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:54:54.299229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-34e9ee8c-eddd-4eed-88cf-59fb72e28486 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:54:54.299229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:54.299229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:54.299229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:54.299229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:54.299229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:54.299229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:54.299229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:54.299229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:54.299229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:54.299229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:54.299229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:54.299229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:54:54.299229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:54:54.299229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:54:54.299229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:54:54.300280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:54:54.300280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:54:54.300280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:54:54.300280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:54:54.300280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:54.300280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:54.300280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:54.300280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:54.300280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:54.300280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:54.300280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:54.300280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:54.300280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:54.300280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:54.300280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:54.300280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:54.301170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:54.301170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:54.301170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:54.301170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:54.301170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:54:54.301170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:54:54.301170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:54:54.301170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:54:54.301170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:54:54.301170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:54:54.301170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:54:54.301170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:54:54.301170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 545/1096] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:54:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:54:54.305012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-2bc4548f-616b-4edd-9339-fd3981386a75 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:54.305012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:54.305012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:54.305012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:54.305012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:54.305012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:54.305012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:54.305012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:54.305012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:54.305012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:54.305012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:54.305012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:54.305012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:54:54.305012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:54:54.305012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:54:54.306363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:54:54.306363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:54.306363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:54.306363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:54.306363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:54.306363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:54.306363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:54.306363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:54.306363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:54.306363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:54.306363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:54.306363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:54.306363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:54.306363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:54.306363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:54.306363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:54.306363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:54.307506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:54:54.307506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:54:54.307506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:54:54.307506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:54.307506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 552/1097] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:54.308140 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9ee8c-eddd-4eed-88cf-59fb72e28486 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:54.312250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4548f-616b-4edd-9339-fd3981386a75 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:54.408846 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e9ee8c-eddd-4eed-88cf-59fb72e28486 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:54.412048 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc4548f-616b-4edd-9339-fd3981386a75 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:54.432670 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:54.444623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e40ba5-8f39-4346-96b4-f2c9a7e5542d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:54.445293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e40ba5-8f39-4346-96b4-f2c9a7e5542d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:54.445573 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e40ba5-8f39-4346-96b4-f2c9a7e5542d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:54.445848 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75e40ba5-8f39-4346-96b4-f2c9a7e5542d None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:54.446629 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e40ba5-8f39-4346-96b4-f2c9a7e5542d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:54.446944 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e40ba5-8f39-4346-96b4-f2c9a7e5542d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:54.447624 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e2458-f97b-44f4-aeb0-da519bdeb8ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:54.448284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e2458-f97b-44f4-aeb0-da519bdeb8ae None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:54.448379 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75e40ba5-8f39-4346-96b4-f2c9a7e5542d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:54.448568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e2458-f97b-44f4-aeb0-da519bdeb8ae None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:54.448842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5e2458-f97b-44f4-aeb0-da519bdeb8ae None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:54.449571 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e2458-f97b-44f4-aeb0-da519bdeb8ae None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:54.449910 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e2458-f97b-44f4-aeb0-da519bdeb8ae None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:54.451296 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee5e2458-f97b-44f4-aeb0-da519bdeb8ae None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:54.456519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 546/1098] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:54:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:54.462136 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e40ba5-8f39-4346-96b4-f2c9a7e5542d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:54.466988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 553/1099] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:54:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:54.467997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e2458-f97b-44f4-aeb0-da519bdeb8ae None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:54.509377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:54.564114 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75e40ba5-8f39-4346-96b4-f2c9a7e5542d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:54.570000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5e2458-f97b-44f4-aeb0-da519bdeb8ae None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:54.597221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ff923-840a-45a0-89ec-381fe7dd1d53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:54.598079 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ff923-840a-45a0-89ec-381fe7dd1d53 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:54.598397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ff923-840a-45a0-89ec-381fe7dd1d53 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:54.598673 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790ff923-840a-45a0-89ec-381fe7dd1d53 None admin] PATH_INFO: `/v3/users/b438c1d1d57f47ddbab2ec188d032872` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:54.599868 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ff923-840a-45a0-89ec-381fe7dd1d53 None admin] RBAC: Authorizing `identity:get_user(user_id=b438c1d1d57f47ddbab2ec188d032872)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:54.601315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790ff923-840a-45a0-89ec-381fe7dd1d53 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:54.602990 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 547/1100] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:54:54 2022] GET /identity/v3/users/b438c1d1d57f47ddbab2ec188d032872 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:54.605125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0f991-d3ae-4cb3-a198-1781fc12b006 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:54.605936 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0f991-d3ae-4cb3-a198-1781fc12b006 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:54.606854 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0f991-d3ae-4cb3-a198-1781fc12b006 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:54.607139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf0f991-d3ae-4cb3-a198-1781fc12b006 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:54.609142 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ff923-840a-45a0-89ec-381fe7dd1d53 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:54.614234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0f991-d3ae-4cb3-a198-1781fc12b006 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:54.615717 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf0f991-d3ae-4cb3-a198-1781fc12b006 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:54.631288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-9cf0f991-d3ae-4cb3-a198-1781fc12b006 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:54.631288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:54:54.631288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:54:54.631288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:54:54.631288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:54:54.631288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:54:54.631288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:54:54.631288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:54:54.631288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:54:54.631288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:54:54.631288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:54:54.631288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:54:54.631288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:54:54.631288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:54:54.631288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:54:54.632403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:54:54.632403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:54:54.632403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:54:54.632403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:54:54.632403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:54:54.632403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:54:54.632403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:54:54.632403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:54:54.632403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:54:54.632403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:54:54.632403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:54:54.632403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:54:54.632403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:54:54.632403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:54:54.632403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:54:54.632403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:54:54.632403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:54:54.633695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:54:54.633695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:54:54.633695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:54:54.633695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:54:54.633695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:54:54.633695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:54:54.633695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:54:54.633695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:54:54.634488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 554/1101] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:54:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:54:54.638194 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0f991-d3ae-4cb3-a198-1781fc12b006 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:54.681394 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:54.744274 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790ff923-840a-45a0-89ec-381fe7dd1d53 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:54.744862 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf0f991-d3ae-4cb3-a198-1781fc12b006 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:54.759834 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:54.782564 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f988123-1fa1-4d86-9150-ec1dc22105c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:54.783225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f988123-1fa1-4d86-9150-ec1dc22105c1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:54.783505 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f988123-1fa1-4d86-9150-ec1dc22105c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:54.783789 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f988123-1fa1-4d86-9150-ec1dc22105c1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:54.794884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74220fb7-a7ab-4b40-a625-25c5b2857ff6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:54.795567 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74220fb7-a7ab-4b40-a625-25c5b2857ff6 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:54.795850 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74220fb7-a7ab-4b40-a625-25c5b2857ff6 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:54.796125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74220fb7-a7ab-4b40-a625-25c5b2857ff6 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:54.796937 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74220fb7-a7ab-4b40-a625-25c5b2857ff6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:54.797280 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74220fb7-a7ab-4b40-a625-25c5b2857ff6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:54.798755 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74220fb7-a7ab-4b40-a625-25c5b2857ff6 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:54.811660 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 555/1102] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:54:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:54.818002 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74220fb7-a7ab-4b40-a625-25c5b2857ff6 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:54.874319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f988123-1fa1-4d86-9150-ec1dc22105c1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:54.875794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f988123-1fa1-4d86-9150-ec1dc22105c1 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:54.922773 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:54.926145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74220fb7-a7ab-4b40-a625-25c5b2857ff6 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:54.961776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040aaa8-f5ee-470c-b58b-600c91a8f04e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:54.962475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040aaa8-f5ee-470c-b58b-600c91a8f04e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:54.962766 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040aaa8-f5ee-470c-b58b-600c91a8f04e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:54.963046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2040aaa8-f5ee-470c-b58b-600c91a8f04e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:54.963894 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040aaa8-f5ee-470c-b58b-600c91a8f04e None admin] RBAC: Adding query filter params (role.id=836753c7d9b44fde817a38c1566d0659, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=ef5dac3f311e43809a46995bab56e780) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:54.964208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040aaa8-f5ee-470c-b58b-600c91a8f04e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:54.966160 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2040aaa8-f5ee-470c-b58b-600c91a8f04e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:54.972336 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 556/1103] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:54:54 2022] GET /identity/v3/role_assignments?user.id=ef5dac3f311e43809a46995bab56e780&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=836753c7d9b44fde817a38c1566d0659 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:54:54.976849 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 548/1104] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:54:54 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 13:54:54.983605 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040aaa8-f5ee-470c-b58b-600c91a8f04e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:55.103803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2040aaa8-f5ee-470c-b58b-600c91a8f04e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:55.147908 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e13073-46a7-4c95-87f4-70323252b7b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:55.148959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e13073-46a7-4c95-87f4-70323252b7b0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:55.149249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e13073-46a7-4c95-87f4-70323252b7b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:55.149598 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e13073-46a7-4c95-87f4-70323252b7b0 None admin] PATH_INFO: `/v3/projects/7eb1e22817e14f85accda9fa9c9aa6e6` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:55.150908 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e13073-46a7-4c95-87f4-70323252b7b0 None admin] RBAC: Authorizing `identity:get_project(project_id=7eb1e22817e14f85accda9fa9c9aa6e6)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:55.152354 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e13073-46a7-4c95-87f4-70323252b7b0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:55.154792 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 557/1105] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:54:54 2022] GET /identity/v3/projects/7eb1e22817e14f85accda9fa9c9aa6e6 => generated 311 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:55.161400 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f988123-1fa1-4d86-9150-ec1dc22105c1 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:55.180537 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:55.275138 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f988123-1fa1-4d86-9150-ec1dc22105c1 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:55.311555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef9b7da-e059-4de8-852b-b9d6555bb3eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:55.312223 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9b7da-e059-4de8-852b-b9d6555bb3eb None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:55.312505 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9b7da-e059-4de8-852b-b9d6555bb3eb None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:55.312784 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef9b7da-e059-4de8-852b-b9d6555bb3eb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:55.313628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9b7da-e059-4de8-852b-b9d6555bb3eb None admin] RBAC: Adding query filter params (role.id=6964639a7f6c41e8a6701ef533151c41, scope.project.id=7eb1e22817e14f85accda9fa9c9aa6e6, user.id=b438c1d1d57f47ddbab2ec188d032872) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:54:55.313952 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9b7da-e059-4de8-852b-b9d6555bb3eb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:55.315418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef9b7da-e059-4de8-852b-b9d6555bb3eb None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:55.321673 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 549/1106] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:54:55 2022] GET /identity/v3/role_assignments?user.id=b438c1d1d57f47ddbab2ec188d032872&scope.project.id=7eb1e22817e14f85accda9fa9c9aa6e6&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 608 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:57.399364 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b48af3-4862-41e4-adf0-f8690534912c None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:57.399910 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b48af3-4862-41e4-adf0-f8690534912c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:57.399910 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43b48af3-4862-41e4-adf0-f8690534912c None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:57.401294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 558/1107] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:57.406413 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52893dd-cadb-4d84-8706-8450d8aba1c0 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:57.406837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52893dd-cadb-4d84-8706-8450d8aba1c0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:57.406986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a52893dd-cadb-4d84-8706-8450d8aba1c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:57.423793 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a52893dd-cadb-4d84-8706-8450d8aba1c0 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:57.424976 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a52893dd-cadb-4d84-8706-8450d8aba1c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:57.491387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 550/1108] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:57 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:57.516345 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b48af3-4862-41e4-adf0-f8690534912c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:57.547600 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:57.618803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43b48af3-4862-41e4-adf0-f8690534912c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:57.652123 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-052d5184-b8c1-4374-8a56-4d2c5992e2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:57.653434 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d5184-b8c1-4374-8a56-4d2c5992e2fb None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:57.653726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d5184-b8c1-4374-8a56-4d2c5992e2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:57.654005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-052d5184-b8c1-4374-8a56-4d2c5992e2fb None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:57.655290 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d5184-b8c1-4374-8a56-4d2c5992e2fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:57.656902 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-052d5184-b8c1-4374-8a56-4d2c5992e2fb None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:57.658703 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 559/1109] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:54:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:54:57.664439 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52893dd-cadb-4d84-8706-8450d8aba1c0 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:54:57.674983 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:57.769864 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a52893dd-cadb-4d84-8706-8450d8aba1c0 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:54:57.806010 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b04e1bd-aef4-4f85-855c-6cdb3385cb19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:54:57.806843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04e1bd-aef4-4f85-855c-6cdb3385cb19 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:57.807129 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04e1bd-aef4-4f85-855c-6cdb3385cb19 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:57.807418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b04e1bd-aef4-4f85-855c-6cdb3385cb19 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:57.808135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04e1bd-aef4-4f85-855c-6cdb3385cb19 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:54:57.809606 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b04e1bd-aef4-4f85-855c-6cdb3385cb19 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:54:57.838353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 551/1110] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 11 13:54:57 2022] POST /identity/v3/users => generated 286 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:54:59.878575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f29aaa-58ea-4943-ad9b-b8f2f32f0fff None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:59.879139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f29aaa-58ea-4943-ad9b-b8f2f32f0fff None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:59.879139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f29aaa-58ea-4943-ad9b-b8f2f32f0fff None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:59.880512 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 560/1111] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:54:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:54:59.885572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485e87a-6e65-45b1-8a5d-31c4e05db876 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:54:59.885957 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485e87a-6e65-45b1-8a5d-31c4e05db876 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:54:59.886129 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5485e87a-6e65-45b1-8a5d-31c4e05db876 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:54:59.902552 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5485e87a-6e65-45b1-8a5d-31c4e05db876 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:54:59.904261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5485e87a-6e65-45b1-8a5d-31c4e05db876 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:54:59.960116 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:54:59.969411 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 552/1112] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:54:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:54:59.992975 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f29aaa-58ea-4943-ad9b-b8f2f32f0fff None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:00.090959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:00.095529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f29aaa-58ea-4943-ad9b-b8f2f32f0fff None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:00.129464 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bacdb6f-5e3c-4f03-be27-ebbd14a133c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:00.130258 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacdb6f-5e3c-4f03-be27-ebbd14a133c5 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:00.130574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacdb6f-5e3c-4f03-be27-ebbd14a133c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:00.130855 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bacdb6f-5e3c-4f03-be27-ebbd14a133c5 None admin] PATH_INFO: `/v3/roles/37f000302df54045892d35f57104660a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:00.132339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bacdb6f-5e3c-4f03-be27-ebbd14a133c5 None admin] RBAC: Authorizing `identity:get_role(role_id=37f000302df54045892d35f57104660a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:00.133701 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bacdb6f-5e3c-4f03-be27-ebbd14a133c5 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:00.135115 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 561/1113] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:54:59 2022] GET /identity/v3/roles/37f000302df54045892d35f57104660a => generated 223 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:00.140945 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485e87a-6e65-45b1-8a5d-31c4e05db876 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:00.237968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5485e87a-6e65-45b1-8a5d-31c4e05db876 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:00.271324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92a2bb-a8e7-4489-9c07-0b3910b1c633 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:00.272179 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92a2bb-a8e7-4489-9c07-0b3910b1c633 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:00.272466 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92a2bb-a8e7-4489-9c07-0b3910b1c633 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:00.272743 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc92a2bb-a8e7-4489-9c07-0b3910b1c633 None admin] PATH_INFO: `/v3/users/cf0429e703eb4753b6c289f283007d7a` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:00.286111 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92a2bb-a8e7-4489-9c07-0b3910b1c633 None admin] RBAC: Authorizing `identity:get_user(user_id=cf0429e703eb4753b6c289f283007d7a)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:00.287584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc92a2bb-a8e7-4489-9c07-0b3910b1c633 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:00.289251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 553/1114] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:55:00 2022] GET /identity/v3/users/cf0429e703eb4753b6c289f283007d7a => generated 286 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:00.295786 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bacdb6f-5e3c-4f03-be27-ebbd14a133c5 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:00.392087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bacdb6f-5e3c-4f03-be27-ebbd14a133c5 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:00.425027 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fffd4-442b-44fc-bebb-fdafcb41b379 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:00.425665 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fffd4-442b-44fc-bebb-fdafcb41b379 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:00.425950 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fffd4-442b-44fc-bebb-fdafcb41b379 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:00.426227 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27fffd4-442b-44fc-bebb-fdafcb41b379 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:00.462811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:00.496270 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fffd4-442b-44fc-bebb-fdafcb41b379 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:00.497736 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27fffd4-442b-44fc-bebb-fdafcb41b379 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:00.603975 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 562/1115] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:55:00 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 13:55:00.609918 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92a2bb-a8e7-4489-9c07-0b3910b1c633 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:00.664584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:00.713563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc92a2bb-a8e7-4489-9c07-0b3910b1c633 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:00.749185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1028b4e9-52ac-4c36-af96-c290b5aca67e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:00.750070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028b4e9-52ac-4c36-af96-c290b5aca67e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:00.750487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028b4e9-52ac-4c36-af96-c290b5aca67e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:00.750668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1028b4e9-52ac-4c36-af96-c290b5aca67e None admin] PATH_INFO: `/v3/projects/7eb1e22817e14f85accda9fa9c9aa6e6` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:00.752010 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028b4e9-52ac-4c36-af96-c290b5aca67e None admin] RBAC: Authorizing `identity:get_project(project_id=7eb1e22817e14f85accda9fa9c9aa6e6)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:00.753692 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1028b4e9-52ac-4c36-af96-c290b5aca67e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:00.755655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 554/1116] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:55:00 2022] GET /identity/v3/projects/7eb1e22817e14f85accda9fa9c9aa6e6 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:00.762052 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fffd4-442b-44fc-bebb-fdafcb41b379 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:00.831912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:00.865705 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27fffd4-442b-44fc-bebb-fdafcb41b379 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:00.903758 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-755ad5d1-d34f-448c-bb8e-6641342ed554 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:00.904437 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ad5d1-d34f-448c-bb8e-6641342ed554 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:00.904697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ad5d1-d34f-448c-bb8e-6641342ed554 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:00.904979 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755ad5d1-d34f-448c-bb8e-6641342ed554 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:00.905826 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ad5d1-d34f-448c-bb8e-6641342ed554 None admin] RBAC: Adding query filter params (role.id=37f000302df54045892d35f57104660a, scope.project.id=7eb1e22817e14f85accda9fa9c9aa6e6, user.id=cf0429e703eb4753b6c289f283007d7a) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:55:00.906142 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ad5d1-d34f-448c-bb8e-6641342ed554 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:00.907615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-755ad5d1-d34f-448c-bb8e-6641342ed554 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:00.913415 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 563/1117] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:55:00 2022] GET /identity/v3/role_assignments?user.id=cf0429e703eb4753b6c289f283007d7a&scope.project.id=7eb1e22817e14f85accda9fa9c9aa6e6&role.id=37f000302df54045892d35f57104660a => generated 259 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:02.916837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8483c11-f3de-4f7f-9b86-11b836c7130d None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:02.917357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8483c11-f3de-4f7f-9b86-11b836c7130d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:02.917470 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8483c11-f3de-4f7f-9b86-11b836c7130d None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:02.918798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 555/1118] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:55:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:55:02.924491 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a404e-9d09-4076-b08e-7a5ff439e8d7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:02.924941 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a404e-9d09-4076-b08e-7a5ff439e8d7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:02.925044 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73a404e-9d09-4076-b08e-7a5ff439e8d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:02.941838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c73a404e-9d09-4076-b08e-7a5ff439e8d7 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:55:02.943065 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c73a404e-9d09-4076-b08e-7a5ff439e8d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:55:03.008555 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 564/1119] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:55:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:55:03.032400 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8483c11-f3de-4f7f-9b86-11b836c7130d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:03.140847 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8483c11-f3de-4f7f-9b86-11b836c7130d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:03.162390 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:03.181300 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3607ae8-4fd0-4b8d-8036-96f46336004e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:03.182072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3607ae8-4fd0-4b8d-8036-96f46336004e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:03.182387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3607ae8-4fd0-4b8d-8036-96f46336004e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:03.182670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3607ae8-4fd0-4b8d-8036-96f46336004e None admin] PATH_INFO: `/v3/roles/37f000302df54045892d35f57104660a` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:03.184257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3607ae8-4fd0-4b8d-8036-96f46336004e None admin] RBAC: Authorizing `identity:get_role(role_id=37f000302df54045892d35f57104660a)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:03.185634 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3607ae8-4fd0-4b8d-8036-96f46336004e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:03.187074 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 556/1120] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:55:03 2022] GET /identity/v3/roles/37f000302df54045892d35f57104660a => generated 223 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:03.192937 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a404e-9d09-4076-b08e-7a5ff439e8d7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:03.257706 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:03.303985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73a404e-9d09-4076-b08e-7a5ff439e8d7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:03.343679 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7b230-4148-49e0-b8c7-9c3223d77ccd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:03.344575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7b230-4148-49e0-b8c7-9c3223d77ccd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:03.344858 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7b230-4148-49e0-b8c7-9c3223d77ccd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:03.345131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd7b230-4148-49e0-b8c7-9c3223d77ccd None admin] PATH_INFO: `/v3/users/cf0429e703eb4753b6c289f283007d7a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:03.346344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7b230-4148-49e0-b8c7-9c3223d77ccd None admin] RBAC: Authorizing `identity:get_user(user_id=cf0429e703eb4753b6c289f283007d7a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:03.347845 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd7b230-4148-49e0-b8c7-9c3223d77ccd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:03.349515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 565/1121] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:55:03 2022] GET /identity/v3/users/cf0429e703eb4753b6c289f283007d7a => generated 286 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:03.355568 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3607ae8-4fd0-4b8d-8036-96f46336004e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:03.458024 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3607ae8-4fd0-4b8d-8036-96f46336004e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:03.493914 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58160183-9028-41a2-a331-b676fdac1707 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:03.494802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58160183-9028-41a2-a331-b676fdac1707 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:03.495091 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58160183-9028-41a2-a331-b676fdac1707 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:03.495369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58160183-9028-41a2-a331-b676fdac1707 None admin] PATH_INFO: `/v3/projects/7eb1e22817e14f85accda9fa9c9aa6e6` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:03.496559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58160183-9028-41a2-a331-b676fdac1707 None admin] RBAC: Authorizing `identity:get_project(project_id=7eb1e22817e14f85accda9fa9c9aa6e6)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:03.498033 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58160183-9028-41a2-a331-b676fdac1707 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:03.499802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 557/1122] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:55:03 2022] GET /identity/v3/projects/7eb1e22817e14f85accda9fa9c9aa6e6 => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:03.506006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7b230-4148-49e0-b8c7-9c3223d77ccd None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:03.606282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd7b230-4148-49e0-b8c7-9c3223d77ccd None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:03.634559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:03.641313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97729739-193f-4768-a47c-2c8069694d0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:03.642205 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97729739-193f-4768-a47c-2c8069694d0e None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:03.642544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97729739-193f-4768-a47c-2c8069694d0e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:03.642827 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97729739-193f-4768-a47c-2c8069694d0e None admin] PATH_INFO: `/v3/projects/7eb1e22817e14f85accda9fa9c9aa6e6/users/cf0429e703eb4753b6c289f283007d7a/roles/37f000302df54045892d35f57104660a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:03.645029 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97729739-193f-4768-a47c-2c8069694d0e None admin] RBAC: Authorizing `identity:create_grant(project_id=7eb1e22817e14f85accda9fa9c9aa6e6, user_id=cf0429e703eb4753b6c289f283007d7a, role_id=37f000302df54045892d35f57104660a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:03.646503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97729739-193f-4768-a47c-2c8069694d0e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:03.661294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 566/1123] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:55:03 2022] PUT /identity/v3/projects/7eb1e22817e14f85accda9fa9c9aa6e6/users/cf0429e703eb4753b6c289f283007d7a/roles/37f000302df54045892d35f57104660a => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:55:05.691300 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757939d-dc18-4ade-bcba-21fa69a98e36 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:05.691963 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757939d-dc18-4ade-bcba-21fa69a98e36 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:05.692257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4757939d-dc18-4ade-bcba-21fa69a98e36 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:05.693659 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 558/1124] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:55:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:55:05.699211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30876a96-5f36-4e7a-87b7-5e8787752e8b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:05.699748 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30876a96-5f36-4e7a-87b7-5e8787752e8b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:05.699842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30876a96-5f36-4e7a-87b7-5e8787752e8b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:05.716872 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-30876a96-5f36-4e7a-87b7-5e8787752e8b None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:55:05.718097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-30876a96-5f36-4e7a-87b7-5e8787752e8b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:55:05.789110 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 567/1125] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:55:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:55:05.814697 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757939d-dc18-4ade-bcba-21fa69a98e36 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:05.874744 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:05.919570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4757939d-dc18-4ade-bcba-21fa69a98e36 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:05.951998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7754b-7397-4e01-94e0-e11d30924c1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:05.952778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7754b-7397-4e01-94e0-e11d30924c1a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:05.953059 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7754b-7397-4e01-94e0-e11d30924c1a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:05.953331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b7754b-7397-4e01-94e0-e11d30924c1a None admin] PATH_INFO: `/v3/roles/37f000302df54045892d35f57104660a` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:05.955015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7754b-7397-4e01-94e0-e11d30924c1a None admin] RBAC: Authorizing `identity:get_role(role_id=37f000302df54045892d35f57104660a)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:05.956377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b7754b-7397-4e01-94e0-e11d30924c1a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:05.957740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 559/1126] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:55:05 2022] GET /identity/v3/roles/37f000302df54045892d35f57104660a => generated 223 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:05.963663 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30876a96-5f36-4e7a-87b7-5e8787752e8b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:06.063159 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30876a96-5f36-4e7a-87b7-5e8787752e8b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:06.083302 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:06.100169 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bfc0d-27cf-44a5-8852-2209867d1000 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:06.101043 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bfc0d-27cf-44a5-8852-2209867d1000 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:06.101329 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bfc0d-27cf-44a5-8852-2209867d1000 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:06.101609 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57bfc0d-27cf-44a5-8852-2209867d1000 None admin] PATH_INFO: `/v3/users/cf0429e703eb4753b6c289f283007d7a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:06.102823 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bfc0d-27cf-44a5-8852-2209867d1000 None admin] RBAC: Authorizing `identity:get_user(user_id=cf0429e703eb4753b6c289f283007d7a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:06.104252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57bfc0d-27cf-44a5-8852-2209867d1000 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:06.106463 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 568/1127] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:55:05 2022] GET /identity/v3/users/cf0429e703eb4753b6c289f283007d7a => generated 286 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:06.112155 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7754b-7397-4e01-94e0-e11d30924c1a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:06.211810 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b7754b-7397-4e01-94e0-e11d30924c1a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:06.235251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:06.245949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c4ea3-45f8-4ebc-ac7c-5a1d18848cff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:06.246750 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c4ea3-45f8-4ebc-ac7c-5a1d18848cff None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:06.247041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c4ea3-45f8-4ebc-ac7c-5a1d18848cff None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:06.247329 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e21c4ea3-45f8-4ebc-ac7c-5a1d18848cff None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:06.315364 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c4ea3-45f8-4ebc-ac7c-5a1d18848cff None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:06.316817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e21c4ea3-45f8-4ebc-ac7c-5a1d18848cff None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:06.438227 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 560/1128] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:55:06 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 13:55:06.445171 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bfc0d-27cf-44a5-8852-2209867d1000 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:06.553787 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57bfc0d-27cf-44a5-8852-2209867d1000 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:06.591939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f38f1e4-01b2-420f-8084-882924c34fd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:06.592816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38f1e4-01b2-420f-8084-882924c34fd9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:06.593104 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38f1e4-01b2-420f-8084-882924c34fd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:06.593388 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f38f1e4-01b2-420f-8084-882924c34fd9 None admin] PATH_INFO: `/v3/projects/7eb1e22817e14f85accda9fa9c9aa6e6` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:06.594650 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38f1e4-01b2-420f-8084-882924c34fd9 None admin] RBAC: Authorizing `identity:get_project(project_id=7eb1e22817e14f85accda9fa9c9aa6e6)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:06.596103 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f38f1e4-01b2-420f-8084-882924c34fd9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:06.597861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 569/1129] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:55:06 2022] GET /identity/v3/projects/7eb1e22817e14f85accda9fa9c9aa6e6 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:06.603985 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c4ea3-45f8-4ebc-ac7c-5a1d18848cff None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:06.628603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:06.709458 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e21c4ea3-45f8-4ebc-ac7c-5a1d18848cff None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:06.743679 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0174d7a6-424b-4ce6-9dea-bf4a52d5e7af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:06.744335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174d7a6-424b-4ce6-9dea-bf4a52d5e7af None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:06.744619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174d7a6-424b-4ce6-9dea-bf4a52d5e7af None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:06.744890 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0174d7a6-424b-4ce6-9dea-bf4a52d5e7af None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:06.745738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0174d7a6-424b-4ce6-9dea-bf4a52d5e7af None admin] RBAC: Adding query filter params (role.id=37f000302df54045892d35f57104660a, scope.project.id=7eb1e22817e14f85accda9fa9c9aa6e6, user.id=cf0429e703eb4753b6c289f283007d7a) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:55:06.746048 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0174d7a6-424b-4ce6-9dea-bf4a52d5e7af None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:06.748079 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0174d7a6-424b-4ce6-9dea-bf4a52d5e7af None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:06.754625 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 561/1130] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:55:06 2022] GET /identity/v3/role_assignments?user.id=cf0429e703eb4753b6c289f283007d7a&scope.project.id=7eb1e22817e14f85accda9fa9c9aa6e6&role.id=37f000302df54045892d35f57104660a => generated 608 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:08.847219 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a85925-27c5-44f8-ae40-e9d12fddb86e None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:08.847747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a85925-27c5-44f8-ae40-e9d12fddb86e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:08.847747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28a85925-27c5-44f8-ae40-e9d12fddb86e None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:08.849143 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 570/1131] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:55:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:55:08.854123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2207833-e60e-4bbe-963f-0b3af45eb51f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:08.854568 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2207833-e60e-4bbe-963f-0b3af45eb51f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:08.854723 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2207833-e60e-4bbe-963f-0b3af45eb51f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:08.871304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b2207833-e60e-4bbe-963f-0b3af45eb51f None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:55:08.872509 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b2207833-e60e-4bbe-963f-0b3af45eb51f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:55:08.938870 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 562/1132] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:55:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:55:08.962989 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a85925-27c5-44f8-ae40-e9d12fddb86e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:09.020817 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:09.065677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28a85925-27c5-44f8-ae40-e9d12fddb86e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:09.099687 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ebb5a90-2ef9-43f9-98ac-57baf12ec07f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:09.100414 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5a90-2ef9-43f9-98ac-57baf12ec07f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:09.100703 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5a90-2ef9-43f9-98ac-57baf12ec07f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:09.100985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ebb5a90-2ef9-43f9-98ac-57baf12ec07f None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:09.102169 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb5a90-2ef9-43f9-98ac-57baf12ec07f None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:09.103631 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ebb5a90-2ef9-43f9-98ac-57baf12ec07f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:09.105297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 571/1133] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:55:08 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:09.111125 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2207833-e60e-4bbe-963f-0b3af45eb51f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:09.129327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:09.208455 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2207833-e60e-4bbe-963f-0b3af45eb51f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:09.240644 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e7257c-5f5f-4401-9ecd-320db19e3104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:09.241296 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7257c-5f5f-4401-9ecd-320db19e3104 None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:09.241581 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7257c-5f5f-4401-9ecd-320db19e3104 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:09.241857 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e7257c-5f5f-4401-9ecd-320db19e3104 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:09.242609 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7257c-5f5f-4401-9ecd-320db19e3104 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:09.244031 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e7257c-5f5f-4401-9ecd-320db19e3104 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:09.269405 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 563/1134] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 11 13:55:09 2022] POST /identity/v3/projects => generated 311 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:55:11.348669 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb620900-827d-4f75-89d0-5b4b223f7b57 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:11.349187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb620900-827d-4f75-89d0-5b4b223f7b57 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:11.349187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb620900-827d-4f75-89d0-5b4b223f7b57 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:11.350562 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 572/1135] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:55:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:55:11.355588 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65957f2-77eb-4c66-a340-066e965bc50a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:11.356121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65957f2-77eb-4c66-a340-066e965bc50a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:11.356121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65957f2-77eb-4c66-a340-066e965bc50a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:11.372994 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b65957f2-77eb-4c66-a340-066e965bc50a None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:55:11.374240 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b65957f2-77eb-4c66-a340-066e965bc50a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:55:11.435888 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:11.441130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 564/1136] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:55:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:55:11.466085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb620900-827d-4f75-89d0-5b4b223f7b57 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:11.567301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb620900-827d-4f75-89d0-5b4b223f7b57 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:11.590024 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:11.601005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f2ad49-cdee-4eab-ac9c-6fe491375ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:11.601724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2ad49-cdee-4eab-ac9c-6fe491375ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:11.602010 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2ad49-cdee-4eab-ac9c-6fe491375ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:11.602287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f2ad49-cdee-4eab-ac9c-6fe491375ab4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:11.603495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f2ad49-cdee-4eab-ac9c-6fe491375ab4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:11.604913 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f2ad49-cdee-4eab-ac9c-6fe491375ab4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:11.607068 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 573/1137] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:55:11 2022] GET /identity/v3/domains/default => generated 199 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:11.612931 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65957f2-77eb-4c66-a340-066e965bc50a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:11.708394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b65957f2-77eb-4c66-a340-066e965bc50a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:11.741352 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faccca6c-a559-4a0d-994b-5100b9595aaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:11.742011 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faccca6c-a559-4a0d-994b-5100b9595aaf None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:11.742318 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faccca6c-a559-4a0d-994b-5100b9595aaf None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:11.742606 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faccca6c-a559-4a0d-994b-5100b9595aaf None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:11.743324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faccca6c-a559-4a0d-994b-5100b9595aaf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:11.744754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faccca6c-a559-4a0d-994b-5100b9595aaf None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:11.764778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 565/1138] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 11 13:55:11 2022] POST /identity/v3/users => generated 286 bytes in 152 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:55:13.894015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4db69-aab4-4432-a865-102bdbe2b8e9 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:13.894675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4db69-aab4-4432-a865-102bdbe2b8e9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:13.894675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6a4db69-aab4-4432-a865-102bdbe2b8e9 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:13.896001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 574/1139] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:55:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:55:13.901674 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fe13f-4939-4d15-81f3-687486f09bd8 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:13.902193 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fe13f-4939-4d15-81f3-687486f09bd8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:13.902362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-264fe13f-4939-4d15-81f3-687486f09bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:13.920068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-264fe13f-4939-4d15-81f3-687486f09bd8 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:55:13.921283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-264fe13f-4939-4d15-81f3-687486f09bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:55:13.975016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:13.993022 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 566/1140] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:55:13 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:55:14.017844 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4db69-aab4-4432-a865-102bdbe2b8e9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:14.148511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6a4db69-aab4-4432-a865-102bdbe2b8e9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:14.191293 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b4eab-25fe-43b5-9594-b14e59e70c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:14.192122 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b4eab-25fe-43b5-9594-b14e59e70c3b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:14.192412 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b4eab-25fe-43b5-9594-b14e59e70c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:14.192693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b97b4eab-25fe-43b5-9594-b14e59e70c3b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:14.205498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b97b4eab-25fe-43b5-9594-b14e59e70c3b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:55:14.213626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-b97b4eab-25fe-43b5-9594-b14e59e70c3b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:55:14.213626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:55:14.213626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:55:14.213626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:55:14.213626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:55:14.213626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:55:14.213626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:55:14.213626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:55:14.213626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:55:14.213626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:55:14.213626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:55:14.213626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:55:14.213626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:55:14.213626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:55:14.213626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:55:14.213626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:55:14.215980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:55:14.215980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:55:14.215980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:55:14.215980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:55:14.215980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:55:14.215980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:55:14.215980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:55:14.215980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:55:14.215980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:55:14.215980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:55:14.215980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:55:14.215980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:55:14.215980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:55:14.215980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:55:14.215980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:55:14.215980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:55:14.216934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:55:14.216934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:55:14.216934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:55:14.216934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:55:14.216934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:55:14.216934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:55:14.216934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:55:14.216934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:55:14.216934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:55:14.216934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:55:14.216934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:55:14.216934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:55:14.216934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 575/1141] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:55:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:55:14.222023 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264fe13f-4939-4d15-81f3-687486f09bd8 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:14.324232 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-264fe13f-4939-4d15-81f3-687486f09bd8 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:14.360372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2fc4b-ec0f-4c4b-8fbf-0553d9d3d1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:14.361188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2fc4b-ec0f-4c4b-8fbf-0553d9d3d1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:14.361475 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2fc4b-ec0f-4c4b-8fbf-0553d9d3d1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:14.361868 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2fc4b-ec0f-4c4b-8fbf-0553d9d3d1e5 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:14.362672 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2fc4b-ec0f-4c4b-8fbf-0553d9d3d1e5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:55:14.362982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2fc4b-ec0f-4c4b-8fbf-0553d9d3d1e5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:14.364654 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2fc4b-ec0f-4c4b-8fbf-0553d9d3d1e5 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:14.374125 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 567/1142] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:55:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:14.380365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b4eab-25fe-43b5-9594-b14e59e70c3b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:14.436750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:14.484785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b97b4eab-25fe-43b5-9594-b14e59e70c3b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:14.521730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ee068-8057-47ef-b44a-63362711f0ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:14.522635 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ee068-8057-47ef-b44a-63362711f0ba None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:14.522925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ee068-8057-47ef-b44a-63362711f0ba None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:14.523204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66ee068-8057-47ef-b44a-63362711f0ba None admin] PATH_INFO: `/v3/users/26b5051542ac4e83a568dba3665a54ef` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:14.538066 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ee068-8057-47ef-b44a-63362711f0ba None admin] RBAC: Authorizing `identity:get_user(user_id=26b5051542ac4e83a568dba3665a54ef)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:14.539757 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66ee068-8057-47ef-b44a-63362711f0ba None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:14.541719 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 576/1143] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:55:14 2022] GET /identity/v3/users/26b5051542ac4e83a568dba3665a54ef => generated 286 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:14.548432 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2fc4b-ec0f-4c4b-8fbf-0553d9d3d1e5 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:14.602370 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:14.661007 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2fc4b-ec0f-4c4b-8fbf-0553d9d3d1e5 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:14.700070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc0e8e-1ca2-446a-ace6-71f14d67ff12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:14.700714 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc0e8e-1ca2-446a-ace6-71f14d67ff12 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:14.701016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc0e8e-1ca2-446a-ace6-71f14d67ff12 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:14.701302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7dc0e8e-1ca2-446a-ace6-71f14d67ff12 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:14.775019 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc0e8e-1ca2-446a-ace6-71f14d67ff12 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:14.776496 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7dc0e8e-1ca2-446a-ace6-71f14d67ff12 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:14.849944 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:14.916183 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 568/1144] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:55:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 368 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 13:55:14.922735 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ee068-8057-47ef-b44a-63362711f0ba None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:15.040512 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66ee068-8057-47ef-b44a-63362711f0ba None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:15.055013 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:15.085548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-038a67bf-9d90-4f46-a5db-155839a6da9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:15.086464 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a67bf-9d90-4f46-a5db-155839a6da9f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:15.086744 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a67bf-9d90-4f46-a5db-155839a6da9f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:15.087019 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-038a67bf-9d90-4f46-a5db-155839a6da9f None admin] PATH_INFO: `/v3/projects/41f90166862147668eda983b8d4ce3af` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:15.088268 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a67bf-9d90-4f46-a5db-155839a6da9f None admin] RBAC: Authorizing `identity:get_project(project_id=41f90166862147668eda983b8d4ce3af)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:15.089701 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-038a67bf-9d90-4f46-a5db-155839a6da9f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:15.092012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 577/1145] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:55:14 2022] GET /identity/v3/projects/41f90166862147668eda983b8d4ce3af => generated 311 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:15.098713 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc0e8e-1ca2-446a-ace6-71f14d67ff12 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:15.209716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7dc0e8e-1ca2-446a-ace6-71f14d67ff12 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:15.246045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de7f7cd5-fa2e-43dd-ba82-c6edd0c45cdf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:15.246739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f7cd5-fa2e-43dd-ba82-c6edd0c45cdf None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:15.247025 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f7cd5-fa2e-43dd-ba82-c6edd0c45cdf None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:15.247301 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de7f7cd5-fa2e-43dd-ba82-c6edd0c45cdf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:15.248150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f7cd5-fa2e-43dd-ba82-c6edd0c45cdf None admin] RBAC: Adding query filter params (role.id=6964639a7f6c41e8a6701ef533151c41, scope.project.id=41f90166862147668eda983b8d4ce3af, user.id=26b5051542ac4e83a568dba3665a54ef) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:55:15.248464 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f7cd5-fa2e-43dd-ba82-c6edd0c45cdf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:15.249922 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de7f7cd5-fa2e-43dd-ba82-c6edd0c45cdf None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:15.256255 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 569/1146] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:55:15 2022] GET /identity/v3/role_assignments?user.id=26b5051542ac4e83a568dba3665a54ef&scope.project.id=41f90166862147668eda983b8d4ce3af&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 259 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:55:17.330414 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad815b-261f-4c03-95e4-a07b3273bf80 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:17.330946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad815b-261f-4c03-95e4-a07b3273bf80 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:17.330946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daad815b-261f-4c03-95e4-a07b3273bf80 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:17.332386 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 578/1147] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:55:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:55:17.338351 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbb330-dc4c-4e11-a909-37298453755b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:17.338833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbb330-dc4c-4e11-a909-37298453755b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:17.338980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cbb330-dc4c-4e11-a909-37298453755b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:17.357026 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-95cbb330-dc4c-4e11-a909-37298453755b None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:55:17.358215 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-95cbb330-dc4c-4e11-a909-37298453755b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:55:17.366251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:17.428143 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 570/1148] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:55:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:55:17.451444 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad815b-261f-4c03-95e4-a07b3273bf80 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:17.564188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daad815b-261f-4c03-95e4-a07b3273bf80 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:17.601667 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13000a6d-b753-4520-95dd-5cfb87744a8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:17.602406 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13000a6d-b753-4520-95dd-5cfb87744a8e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:17.602698 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13000a6d-b753-4520-95dd-5cfb87744a8e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:17.602970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13000a6d-b753-4520-95dd-5cfb87744a8e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:17.613811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13000a6d-b753-4520-95dd-5cfb87744a8e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:55:17.621633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-13000a6d-b753-4520-95dd-5cfb87744a8e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:55:17.621633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:55:17.621633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:55:17.621633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:55:17.621633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:55:17.621633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:55:17.621633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:55:17.621633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:55:17.621633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:55:17.621633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:55:17.621633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:55:17.621633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:55:17.621633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:55:17.621633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:55:17.621633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:55:17.621633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:55:17.622842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:55:17.622842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:55:17.622842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:55:17.622842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:55:17.622842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:55:17.622842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:55:17.622842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:55:17.622842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:55:17.622842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:55:17.622842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:55:17.622842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:55:17.622842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:55:17.622842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:55:17.622842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:55:17.622842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:55:17.622842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:55:17.623709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:55:17.623709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:55:17.623709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:55:17.623709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:55:17.623709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:55:17.623709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:55:17.623709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:55:17.623709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:55:17.623709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:55:17.623709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:55:17.623709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:55:17.623709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:55:17.623709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 579/1149] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:55:17 2022] GET /identity/v3/roles/admin => generated 83 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:55:17.629348 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbb330-dc4c-4e11-a909-37298453755b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:17.737549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cbb330-dc4c-4e11-a909-37298453755b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:17.775311 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e5a67-81b6-45fa-b92f-06c893c8059a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:17.776000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e5a67-81b6-45fa-b92f-06c893c8059a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:17.776288 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e5a67-81b6-45fa-b92f-06c893c8059a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:17.776569 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5e5a67-81b6-45fa-b92f-06c893c8059a None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:17.777935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e5a67-81b6-45fa-b92f-06c893c8059a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:55:17.778254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e5a67-81b6-45fa-b92f-06c893c8059a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:17.779739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5e5a67-81b6-45fa-b92f-06c893c8059a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:17.789550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 571/1150] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:55:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:17.797144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13000a6d-b753-4520-95dd-5cfb87744a8e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:17.843862 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:17.907484 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13000a6d-b753-4520-95dd-5cfb87744a8e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:17.947402 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184200f-4e8f-4b8b-8b4d-c81ca8c44d39 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:17.948323 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184200f-4e8f-4b8b-8b4d-c81ca8c44d39 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:17.948604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184200f-4e8f-4b8b-8b4d-c81ca8c44d39 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:17.948879 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184200f-4e8f-4b8b-8b4d-c81ca8c44d39 None admin] PATH_INFO: `/v3/users/26b5051542ac4e83a568dba3665a54ef` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:17.951121 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184200f-4e8f-4b8b-8b4d-c81ca8c44d39 None admin] RBAC: Authorizing `identity:get_user(user_id=26b5051542ac4e83a568dba3665a54ef)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:17.952579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9184200f-4e8f-4b8b-8b4d-c81ca8c44d39 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:17.954273 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 580/1151] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:55:17 2022] GET /identity/v3/users/26b5051542ac4e83a568dba3665a54ef => generated 286 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:17.960387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e5a67-81b6-45fa-b92f-06c893c8059a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:17.964321 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:18.062037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5e5a67-81b6-45fa-b92f-06c893c8059a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:18.096879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7c7ec0-3a62-41e1-9435-ab71cad87f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:18.097741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c7ec0-3a62-41e1-9435-ab71cad87f0b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:18.098030 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c7ec0-3a62-41e1-9435-ab71cad87f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:18.098324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7c7ec0-3a62-41e1-9435-ab71cad87f0b None admin] PATH_INFO: `/v3/projects/41f90166862147668eda983b8d4ce3af` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:18.099540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c7ec0-3a62-41e1-9435-ab71cad87f0b None admin] RBAC: Authorizing `identity:get_project(project_id=41f90166862147668eda983b8d4ce3af)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:18.100973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7c7ec0-3a62-41e1-9435-ab71cad87f0b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:18.102715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 572/1152] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:55:17 2022] GET /identity/v3/projects/41f90166862147668eda983b8d4ce3af => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:18.108808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184200f-4e8f-4b8b-8b4d-c81ca8c44d39 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:18.216103 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184200f-4e8f-4b8b-8b4d-c81ca8c44d39 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:18.223986 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:18.252202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2861ab-5258-4d68-9d5f-249dfdc74b7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:18.253099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2861ab-5258-4d68-9d5f-249dfdc74b7f None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:18.253387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2861ab-5258-4d68-9d5f-249dfdc74b7f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:18.253665 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2861ab-5258-4d68-9d5f-249dfdc74b7f None admin] PATH_INFO: `/v3/projects/41f90166862147668eda983b8d4ce3af/users/26b5051542ac4e83a568dba3665a54ef/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:18.255719 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2861ab-5258-4d68-9d5f-249dfdc74b7f None admin] RBAC: Authorizing `identity:create_grant(project_id=41f90166862147668eda983b8d4ce3af, user_id=26b5051542ac4e83a568dba3665a54ef, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:18.257178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2861ab-5258-4d68-9d5f-249dfdc74b7f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:18.272516 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 581/1153] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:55:18 2022] PUT /identity/v3/projects/41f90166862147668eda983b8d4ce3af/users/26b5051542ac4e83a568dba3665a54ef/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 13:55:20.374386 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c72f-b5c9-4b2e-8c08-23a29cefaba4 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:20.374932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c72f-b5c9-4b2e-8c08-23a29cefaba4 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:20.374932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54c4c72f-b5c9-4b2e-8c08-23a29cefaba4 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:20.376288 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 573/1154] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:55:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:55:20.382223 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87aa535-028c-4167-8f1d-3443185fc638 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:20.382803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87aa535-028c-4167-8f1d-3443185fc638 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:20.382803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87aa535-028c-4167-8f1d-3443185fc638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:20.401773 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c87aa535-028c-4167-8f1d-3443185fc638 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:55:20.403643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c87aa535-028c-4167-8f1d-3443185fc638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:55:20.479479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 582/1155] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:55:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:55:20.504493 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4c72f-b5c9-4b2e-8c08-23a29cefaba4 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:20.588285 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:20.621798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54c4c72f-b5c9-4b2e-8c08-23a29cefaba4 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:20.659461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d371cb48-4766-4553-a4d6-646bcb37ea74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:20.660265 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371cb48-4766-4553-a4d6-646bcb37ea74 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:20.660551 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371cb48-4766-4553-a4d6-646bcb37ea74 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:20.660829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d371cb48-4766-4553-a4d6-646bcb37ea74 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:20.672243 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d371cb48-4766-4553-a4d6-646bcb37ea74 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:55:20.680304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-d371cb48-4766-4553-a4d6-646bcb37ea74 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:55:20.680304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:55:20.680304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:55:20.680304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:55:20.680304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:55:20.680304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:55:20.680304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:55:20.680304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:55:20.680304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:55:20.680304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:55:20.680304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:55:20.680304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:55:20.680304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:55:20.680304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:55:20.680304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:55:20.680304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:55:20.681436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:55:20.681436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:55:20.681436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:55:20.681436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:55:20.681436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:55:20.681436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:55:20.681436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:55:20.681436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:55:20.681436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:55:20.681436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:55:20.681436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:55:20.681436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:55:20.681436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:55:20.681436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:55:20.681436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:55:20.681436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:55:20.682394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:55:20.682394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:55:20.682394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:55:20.682394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:55:20.682394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:55:20.682394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:55:20.682394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:55:20.682394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:55:20.682394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:55:20.682394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:55:20.682394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:55:20.682394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:55:20.682394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 574/1156] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:55:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:55:20.687889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87aa535-028c-4167-8f1d-3443185fc638 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:20.786220 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:20.792783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87aa535-028c-4167-8f1d-3443185fc638 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:20.828117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f8b37-54ba-40f8-b4bd-4a8f786cac26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:20.828795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f8b37-54ba-40f8-b4bd-4a8f786cac26 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:20.829081 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f8b37-54ba-40f8-b4bd-4a8f786cac26 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:20.829362 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd7f8b37-54ba-40f8-b4bd-4a8f786cac26 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:20.830124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f8b37-54ba-40f8-b4bd-4a8f786cac26 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:55:20.830468 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f8b37-54ba-40f8-b4bd-4a8f786cac26 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:20.831843 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd7f8b37-54ba-40f8-b4bd-4a8f786cac26 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:20.841005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 583/1157] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:55:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:20.846911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d371cb48-4766-4553-a4d6-646bcb37ea74 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:20.948083 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d371cb48-4766-4553-a4d6-646bcb37ea74 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:20.984791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:20.990480 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12283ff-4ff9-4596-9292-230efdec06d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:20.991510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12283ff-4ff9-4596-9292-230efdec06d7 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:20.992007 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12283ff-4ff9-4596-9292-230efdec06d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:20.992291 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12283ff-4ff9-4596-9292-230efdec06d7 None admin] PATH_INFO: `/v3/users/26b5051542ac4e83a568dba3665a54ef` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:20.993524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12283ff-4ff9-4596-9292-230efdec06d7 None admin] RBAC: Authorizing `identity:get_user(user_id=26b5051542ac4e83a568dba3665a54ef)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:20.995009 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12283ff-4ff9-4596-9292-230efdec06d7 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:20.997187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 575/1158] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:55:20 2022] GET /identity/v3/users/26b5051542ac4e83a568dba3665a54ef => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:21.003142 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f8b37-54ba-40f8-b4bd-4a8f786cac26 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:21.111881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd7f8b37-54ba-40f8-b4bd-4a8f786cac26 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:21.147586 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01191a50-7913-4a08-b791-bf23f764227e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:21.148249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01191a50-7913-4a08-b791-bf23f764227e None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:21.148537 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01191a50-7913-4a08-b791-bf23f764227e None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:21.148816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01191a50-7913-4a08-b791-bf23f764227e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:21.176502 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:21.221726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01191a50-7913-4a08-b791-bf23f764227e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:21.223120 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01191a50-7913-4a08-b791-bf23f764227e None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:21.330594 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 584/1159] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:55:21 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 13:55:21.336932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12283ff-4ff9-4596-9292-230efdec06d7 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:21.457462 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12283ff-4ff9-4596-9292-230efdec06d7 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:21.492814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64585134-7773-45b6-9a2d-18aa4c0b0151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:21.493623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64585134-7773-45b6-9a2d-18aa4c0b0151 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:21.493965 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64585134-7773-45b6-9a2d-18aa4c0b0151 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:21.494280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64585134-7773-45b6-9a2d-18aa4c0b0151 None admin] PATH_INFO: `/v3/projects/41f90166862147668eda983b8d4ce3af` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:21.495797 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64585134-7773-45b6-9a2d-18aa4c0b0151 None admin] RBAC: Authorizing `identity:get_project(project_id=41f90166862147668eda983b8d4ce3af)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:21.497385 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64585134-7773-45b6-9a2d-18aa4c0b0151 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:21.499757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 576/1160] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:55:21 2022] GET /identity/v3/projects/41f90166862147668eda983b8d4ce3af => generated 311 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:21.508240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01191a50-7913-4a08-b791-bf23f764227e None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:21.570961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:21.616907 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01191a50-7913-4a08-b791-bf23f764227e None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:21.654971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a0ba50-6d9e-49c0-bef7-72eb7eff36f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:21.655660 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0ba50-6d9e-49c0-bef7-72eb7eff36f9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:21.655946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0ba50-6d9e-49c0-bef7-72eb7eff36f9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:21.656223 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a0ba50-6d9e-49c0-bef7-72eb7eff36f9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:21.657096 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0ba50-6d9e-49c0-bef7-72eb7eff36f9 None admin] RBAC: Adding query filter params (role.id=6964639a7f6c41e8a6701ef533151c41, scope.project.id=41f90166862147668eda983b8d4ce3af, user.id=26b5051542ac4e83a568dba3665a54ef) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:55:21.657414 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0ba50-6d9e-49c0-bef7-72eb7eff36f9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:21.658924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a0ba50-6d9e-49c0-bef7-72eb7eff36f9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:21.665760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 585/1161] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:55:21 2022] GET /identity/v3/role_assignments?user.id=26b5051542ac4e83a568dba3665a54ef&scope.project.id=41f90166862147668eda983b8d4ce3af&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 608 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:55:23.814065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bc3f1-5595-442e-9a6d-09cb73442ce0 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:23.814716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bc3f1-5595-442e-9a6d-09cb73442ce0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:23.814716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358bc3f1-5595-442e-9a6d-09cb73442ce0 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:23.816038 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 577/1162] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:55:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:55:23.823488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384ce90-4d36-4294-b57c-5efbd27f67e8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:23.824581 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384ce90-4d36-4294-b57c-5efbd27f67e8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:23.824581 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9384ce90-4d36-4294-b57c-5efbd27f67e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:23.842899 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9384ce90-4d36-4294-b57c-5efbd27f67e8 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:55:23.844127 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9384ce90-4d36-4294-b57c-5efbd27f67e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:55:23.922633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 586/1163] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:55:23 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:55:23.946243 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bc3f1-5595-442e-9a6d-09cb73442ce0 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:24.025506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:24.075079 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358bc3f1-5595-442e-9a6d-09cb73442ce0 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:24.114880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8491232-189f-495f-830a-10270c91363a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:24.115543 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8491232-189f-495f-830a-10270c91363a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:24.115823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8491232-189f-495f-830a-10270c91363a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:24.116091 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8491232-189f-495f-830a-10270c91363a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:24.192756 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8491232-189f-495f-830a-10270c91363a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:24.194252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8491232-189f-495f-830a-10270c91363a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:24.262670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:24.309133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 578/1164] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:55:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 13:55:24.317426 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384ce90-4d36-4294-b57c-5efbd27f67e8 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:24.369843 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:24.418396 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9384ce90-4d36-4294-b57c-5efbd27f67e8 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:24.456072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a538c5-a55f-4938-b1c1-c9962d678181 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:24.456876 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a538c5-a55f-4938-b1c1-c9962d678181 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:24.457162 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a538c5-a55f-4938-b1c1-c9962d678181 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:24.457436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a538c5-a55f-4938-b1c1-c9962d678181 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:24.464473 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a538c5-a55f-4938-b1c1-c9962d678181 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:24.465914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a538c5-a55f-4938-b1c1-c9962d678181 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:24.480640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-d4a538c5-a55f-4938-b1c1-c9962d678181 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 11 13:55:24.480640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:55:24.480640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 11 13:55:24.480640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 11 13:55:24.480640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:55:24.480640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:55:24.480640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:55:24.480640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:55:24.480640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 11 13:55:24.480640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:55:24.480640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:55:24.480640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:55:24.480640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:55:24.480640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:55:24.480640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:55:24.480640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:55:24.480640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:55:24.481914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:55:24.481914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:55:24.481914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:55:24.481914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:55:24.481914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:55:24.481914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:55:24.481914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:55:24.481914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:55:24.481914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:55:24.481914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:55:24.481914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:55:24.481914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:55:24.481914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:55:24.481914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:55:24.481914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:55:24.481914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:55:24.482887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:55:24.482887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:55:24.482887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:55:24.482887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:55:24.482887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:55:24.482887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:55:24.482887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:55:24.482887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:55:24.482887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:55:24.482887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:55:24.482887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 11 13:55:24.482887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:55:24.482887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 11 13:55:24.482887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:55:24.482887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 587/1165] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 11 13:55:24 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:55:24.487842 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8491232-189f-495f-830a-10270c91363a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:24.584626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8491232-189f-495f-830a-10270c91363a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:24.618378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40bfc8d3-75c8-4e6d-8846-e29f730769c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:24.619053 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bfc8d3-75c8-4e6d-8846-e29f730769c1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:24.619343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bfc8d3-75c8-4e6d-8846-e29f730769c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:24.619624 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bfc8d3-75c8-4e6d-8846-e29f730769c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:24.620436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bfc8d3-75c8-4e6d-8846-e29f730769c1 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:55:24.620780 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bfc8d3-75c8-4e6d-8846-e29f730769c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:24.622785 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40bfc8d3-75c8-4e6d-8846-e29f730769c1 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:24.630148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 579/1166] 10.222.0.49 () {56 vars in 1115 bytes} [Mon Jul 11 13:55:24 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:26.651261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b4027-96ee-4ab6-b689-77865d4de712 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:26.651818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b4027-96ee-4ab6-b689-77865d4de712 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:26.651818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3b4027-96ee-4ab6-b689-77865d4de712 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:26.653821 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 588/1167] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:55:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:55:26.659051 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004af524-7d30-4cf5-b4fd-c684cc81e570 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:26.659446 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004af524-7d30-4cf5-b4fd-c684cc81e570 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:26.659617 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004af524-7d30-4cf5-b4fd-c684cc81e570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:26.676378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-004af524-7d30-4cf5-b4fd-c684cc81e570 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:55:26.677577 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-004af524-7d30-4cf5-b4fd-c684cc81e570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:55:26.704550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:26.746527 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 580/1168] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:55:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:55:26.770271 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b4027-96ee-4ab6-b689-77865d4de712 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:26.881358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3b4027-96ee-4ab6-b689-77865d4de712 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:26.895185 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:26.916880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c40cadab-d4ba-40c1-b3f6-59765edbadda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:26.917502 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cadab-d4ba-40c1-b3f6-59765edbadda None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:26.917792 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cadab-d4ba-40c1-b3f6-59765edbadda None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:26.918071 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c40cadab-d4ba-40c1-b3f6-59765edbadda None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:26.918852 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40cadab-d4ba-40c1-b3f6-59765edbadda None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:26.920209 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c40cadab-d4ba-40c1-b3f6-59765edbadda None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:26.935576 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 589/1169] 10.222.0.49 () {60 vars in 1138 bytes} [Mon Jul 11 13:55:26 2022] POST /identity/v3/domains => generated 267 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:55:28.999459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f757a-1d84-4859-aa79-ad000acdbb15 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:28.999999 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f757a-1d84-4859-aa79-ad000acdbb15 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:28.999999 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f47f757a-1d84-4859-aa79-ad000acdbb15 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:29.001414 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 581/1170] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:55:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:55:29.006812 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f2575-7726-458a-9e8c-4c8dfeeb1ff0 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:29.007263 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f2575-7726-458a-9e8c-4c8dfeeb1ff0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:29.007373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845f2575-7726-458a-9e8c-4c8dfeeb1ff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:29.028282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-845f2575-7726-458a-9e8c-4c8dfeeb1ff0 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:55:29.029505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-845f2575-7726-458a-9e8c-4c8dfeeb1ff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:55:29.105647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 590/1171] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:55:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:55:29.129912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f757a-1d84-4859-aa79-ad000acdbb15 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:29.241154 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f47f757a-1d84-4859-aa79-ad000acdbb15 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:29.273369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b615cc40-ba3e-4a2a-b169-02a3b400743b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:29.274495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615cc40-ba3e-4a2a-b169-02a3b400743b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:29.274495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615cc40-ba3e-4a2a-b169-02a3b400743b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:29.274790 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b615cc40-ba3e-4a2a-b169-02a3b400743b None admin] PATH_INFO: `/v3/domains/739194f27b73438baad6342bf198fe2c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:29.285587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615cc40-ba3e-4a2a-b169-02a3b400743b None admin] RBAC: Authorizing `identity:get_domain(domain_id=739194f27b73438baad6342bf198fe2c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:29.287060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b615cc40-ba3e-4a2a-b169-02a3b400743b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:29.288716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 582/1172] 10.222.0.49 () {56 vars in 1183 bytes} [Mon Jul 11 13:55:29 2022] GET /identity/v3/domains/739194f27b73438baad6342bf198fe2c => generated 267 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:55:29.295181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f2575-7726-458a-9e8c-4c8dfeeb1ff0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:29.393792 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845f2575-7726-458a-9e8c-4c8dfeeb1ff0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:29.395398 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:29.429127 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc88f7da-4de0-4e36-934d-9d160e31b670 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:29.430038 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88f7da-4de0-4e36-934d-9d160e31b670 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:29.430346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88f7da-4de0-4e36-934d-9d160e31b670 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:29.430629 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc88f7da-4de0-4e36-934d-9d160e31b670 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:29.431343 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88f7da-4de0-4e36-934d-9d160e31b670 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:29.432768 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc88f7da-4de0-4e36-934d-9d160e31b670 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:29.458066 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 591/1173] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 11 13:55:29 2022] POST /identity/v3/projects => generated 361 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 13:55:31.516670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f70794-12e1-4d49-aca3-fa64645f0288 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:31.517226 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f70794-12e1-4d49-aca3-fa64645f0288 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:31.517226 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f70794-12e1-4d49-aca3-fa64645f0288 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:31.518674 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 583/1174] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:55:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:55:31.523939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828585ca-f843-41dd-aaae-268b250601f2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:31.524387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828585ca-f843-41dd-aaae-268b250601f2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:31.524489 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-828585ca-f843-41dd-aaae-268b250601f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:31.541224 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-828585ca-f843-41dd-aaae-268b250601f2 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:55:31.542452 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-828585ca-f843-41dd-aaae-268b250601f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:55:31.614126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 592/1175] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:55:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:55:31.639722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f70794-12e1-4d49-aca3-fa64645f0288 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:31.667398 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:31.752802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f70794-12e1-4d49-aca3-fa64645f0288 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:31.791963 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7068319-884c-4d91-8eeb-d6432380918a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:31.793275 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7068319-884c-4d91-8eeb-d6432380918a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:31.793560 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7068319-884c-4d91-8eeb-d6432380918a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:31.793842 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7068319-884c-4d91-8eeb-d6432380918a None admin] PATH_INFO: `/v3/domains/739194f27b73438baad6342bf198fe2c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:31.795109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7068319-884c-4d91-8eeb-d6432380918a None admin] RBAC: Authorizing `identity:get_domain(domain_id=739194f27b73438baad6342bf198fe2c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:31.796580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7068319-884c-4d91-8eeb-d6432380918a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:31.798258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 584/1176] 10.222.0.49 () {56 vars in 1183 bytes} [Mon Jul 11 13:55:31 2022] GET /identity/v3/domains/739194f27b73438baad6342bf198fe2c => generated 267 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:31.804517 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828585ca-f843-41dd-aaae-268b250601f2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:31.891491 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:31.908534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-828585ca-f843-41dd-aaae-268b250601f2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:31.943946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f5c81e-29f9-4a5e-bc65-a8b4877d2619 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:31.944632 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5c81e-29f9-4a5e-bc65-a8b4877d2619 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:31.944915 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5c81e-29f9-4a5e-bc65-a8b4877d2619 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:31.945197 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f5c81e-29f9-4a5e-bc65-a8b4877d2619 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:31.946050 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5c81e-29f9-4a5e-bc65-a8b4877d2619 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:31.947548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f5c81e-29f9-4a5e-bc65-a8b4877d2619 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:31.968902 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 593/1177] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 11 13:55:31 2022] POST /identity/v3/users => generated 311 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:55:34.059502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f97cce-26eb-4c1f-a818-14e827c54481 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:34.060017 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f97cce-26eb-4c1f-a818-14e827c54481 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:34.060111 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92f97cce-26eb-4c1f-a818-14e827c54481 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:34.061437 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 585/1178] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:55:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:55:34.066787 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6493df-8445-4733-aac7-ae5752c70243 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:34.067205 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6493df-8445-4733-aac7-ae5752c70243 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:34.067342 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6493df-8445-4733-aac7-ae5752c70243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:34.084902 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ea6493df-8445-4733-aac7-ae5752c70243 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:55:34.086188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ea6493df-8445-4733-aac7-ae5752c70243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:55:34.169943 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 594/1179] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:55:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:55:34.194190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f97cce-26eb-4c1f-a818-14e827c54481 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:34.286328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:34.296870 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92f97cce-26eb-4c1f-a818-14e827c54481 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:34.330331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe27cc-e9b8-4ecc-805c-9b6f9943b43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:34.331152 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe27cc-e9b8-4ecc-805c-9b6f9943b43e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:34.331343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe27cc-e9b8-4ecc-805c-9b6f9943b43e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:34.331619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fe27cc-e9b8-4ecc-805c-9b6f9943b43e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:34.342234 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fe27cc-e9b8-4ecc-805c-9b6f9943b43e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:55:34.349838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-a2fe27cc-e9b8-4ecc-805c-9b6f9943b43e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:55:34.349838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:55:34.349838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:55:34.349838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:55:34.349838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:55:34.349838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:55:34.349838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:55:34.349838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:55:34.349838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:55:34.349838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:55:34.349838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:55:34.349838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:55:34.349838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:55:34.349838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:55:34.349838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:55:34.349838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:55:34.351045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:55:34.351045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:55:34.351045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:55:34.351045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:55:34.351045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:55:34.351045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:55:34.351045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:55:34.351045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:55:34.351045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:55:34.351045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:55:34.351045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:55:34.351045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:55:34.351045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:55:34.351045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:55:34.351045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:55:34.351045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:55:34.351972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:55:34.351972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:55:34.351972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:55:34.351972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:55:34.351972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:55:34.351972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:55:34.351972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:55:34.351972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:55:34.351972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:55:34.351972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:55:34.351972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:55:34.351972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:55:34.351972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 586/1180] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:55:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:55:34.357153 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6493df-8445-4733-aac7-ae5752c70243 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:34.424912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:34.458032 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6493df-8445-4733-aac7-ae5752c70243 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:34.491445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93018a05-1417-453f-a04a-2d2cf4998016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:34.492135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93018a05-1417-453f-a04a-2d2cf4998016 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:34.492424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93018a05-1417-453f-a04a-2d2cf4998016 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:34.492704 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93018a05-1417-453f-a04a-2d2cf4998016 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:34.493469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93018a05-1417-453f-a04a-2d2cf4998016 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:55:34.493782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93018a05-1417-453f-a04a-2d2cf4998016 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:34.495211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93018a05-1417-453f-a04a-2d2cf4998016 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:34.504795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 595/1181] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:55:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:34.510571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe27cc-e9b8-4ecc-805c-9b6f9943b43e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:34.608551 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fe27cc-e9b8-4ecc-805c-9b6f9943b43e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:34.641846 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fa412-50bf-4301-a109-b5306009b1a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:34.642736 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fa412-50bf-4301-a109-b5306009b1a2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:34.643025 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fa412-50bf-4301-a109-b5306009b1a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:34.643303 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552fa412-50bf-4301-a109-b5306009b1a2 None admin] PATH_INFO: `/v3/users/416de5d8cfb04f7a822a99268a89c959` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:34.656187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552fa412-50bf-4301-a109-b5306009b1a2 None admin] RBAC: Authorizing `identity:get_user(user_id=416de5d8cfb04f7a822a99268a89c959)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:34.657670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-552fa412-50bf-4301-a109-b5306009b1a2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:34.659374 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 587/1182] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:55:34 2022] GET /identity/v3/users/416de5d8cfb04f7a822a99268a89c959 => generated 311 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:34.665203 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93018a05-1417-453f-a04a-2d2cf4998016 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:34.763841 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93018a05-1417-453f-a04a-2d2cf4998016 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:34.792988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:34.798115 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fcd0d7-67aa-44fb-be3e-9daa2e982ece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:34.798790 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcd0d7-67aa-44fb-be3e-9daa2e982ece None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:34.799074 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcd0d7-67aa-44fb-be3e-9daa2e982ece None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:34.799354 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56fcd0d7-67aa-44fb-be3e-9daa2e982ece None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:34.864992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fcd0d7-67aa-44fb-be3e-9daa2e982ece None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:34.866381 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56fcd0d7-67aa-44fb-be3e-9daa2e982ece None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:34.961441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 596/1183] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:55:34 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 13:55:34.967343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fa412-50bf-4301-a109-b5306009b1a2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:34.977877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:35.069205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552fa412-50bf-4301-a109-b5306009b1a2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:35.103256 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98ba4657-d583-4c4a-b5cf-c7ed23fe675a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:35.104109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba4657-d583-4c4a-b5cf-c7ed23fe675a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:35.104394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba4657-d583-4c4a-b5cf-c7ed23fe675a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:35.105357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ba4657-d583-4c4a-b5cf-c7ed23fe675a None admin] PATH_INFO: `/v3/projects/e0eebdae09464776bfd0e92e5c017e3e` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:35.106874 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba4657-d583-4c4a-b5cf-c7ed23fe675a None admin] RBAC: Authorizing `identity:get_project(project_id=e0eebdae09464776bfd0e92e5c017e3e)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:35.108353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98ba4657-d583-4c4a-b5cf-c7ed23fe675a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:35.110090 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 588/1184] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:55:34 2022] GET /identity/v3/projects/e0eebdae09464776bfd0e92e5c017e3e => generated 361 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:35.116330 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fcd0d7-67aa-44fb-be3e-9daa2e982ece None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:35.146599 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:35.219036 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56fcd0d7-67aa-44fb-be3e-9daa2e982ece None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:35.252896 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4de72f-fcde-45ee-bc5b-260a46a2a720 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:35.253585 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4de72f-fcde-45ee-bc5b-260a46a2a720 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:35.253847 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4de72f-fcde-45ee-bc5b-260a46a2a720 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:35.254122 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4de72f-fcde-45ee-bc5b-260a46a2a720 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:35.254997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4de72f-fcde-45ee-bc5b-260a46a2a720 None admin] RBAC: Adding query filter params (role.id=6964639a7f6c41e8a6701ef533151c41, scope.project.id=e0eebdae09464776bfd0e92e5c017e3e, user.id=416de5d8cfb04f7a822a99268a89c959) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:55:35.255310 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4de72f-fcde-45ee-bc5b-260a46a2a720 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:35.256742 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e4de72f-fcde-45ee-bc5b-260a46a2a720 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:35.263072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 597/1185] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:55:35 2022] GET /identity/v3/role_assignments?user.id=416de5d8cfb04f7a822a99268a89c959&scope.project.id=e0eebdae09464776bfd0e92e5c017e3e&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 259 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:37.277546 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877d36e-884d-4d7c-8080-72dcf9b742e2 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:37.278137 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877d36e-884d-4d7c-8080-72dcf9b742e2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:37.278284 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c877d36e-884d-4d7c-8080-72dcf9b742e2 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:37.279914 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 589/1186] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:55:37 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:55:37.285943 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c43349-98ff-481d-b502-05a0a74ff989 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:37.286488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c43349-98ff-481d-b502-05a0a74ff989 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:37.286601 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c43349-98ff-481d-b502-05a0a74ff989 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:37.303617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d6c43349-98ff-481d-b502-05a0a74ff989 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:55:37.305075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d6c43349-98ff-481d-b502-05a0a74ff989 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:55:37.372028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 598/1187] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:55:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:55:37.394164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877d36e-884d-4d7c-8080-72dcf9b742e2 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:37.469006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:37.492690 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c877d36e-884d-4d7c-8080-72dcf9b742e2 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:37.537065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cdfb5-f937-460b-ae8d-3095c9e096b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:37.537862 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cdfb5-f937-460b-ae8d-3095c9e096b6 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:37.538148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cdfb5-f937-460b-ae8d-3095c9e096b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:37.538455 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9cdfb5-f937-460b-ae8d-3095c9e096b6 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:37.549170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb9cdfb5-f937-460b-ae8d-3095c9e096b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:55:37.557041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-fb9cdfb5-f937-460b-ae8d-3095c9e096b6 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:55:37.557041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:55:37.557041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:55:37.557041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:55:37.557041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:55:37.557041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:55:37.557041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:55:37.557041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:55:37.557041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:55:37.557041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:55:37.557041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:55:37.557041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:55:37.557041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:55:37.557041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:55:37.557041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:55:37.557041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:55:37.558206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:55:37.558206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:55:37.558206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:55:37.558206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:55:37.558206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:55:37.558206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:55:37.558206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:55:37.558206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:55:37.558206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:55:37.558206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:55:37.558206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:55:37.558206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:55:37.558206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:55:37.558206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:55:37.558206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:55:37.558206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:55:37.559139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:55:37.559139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:55:37.559139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:55:37.559139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:55:37.559139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:55:37.559139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:55:37.559139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:55:37.559139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:55:37.559139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:55:37.559139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:55:37.559139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:55:37.559139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:55:37.559139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 590/1188] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:55:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:55:37.565120 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c43349-98ff-481d-b502-05a0a74ff989 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:37.594197 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:37.669824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c43349-98ff-481d-b502-05a0a74ff989 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:37.704117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7ebfc-03be-49c8-ae58-8e1632323b5b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:37.704781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7ebfc-03be-49c8-ae58-8e1632323b5b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:37.705065 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7ebfc-03be-49c8-ae58-8e1632323b5b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:37.705338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d7ebfc-03be-49c8-ae58-8e1632323b5b None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:37.706089 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7ebfc-03be-49c8-ae58-8e1632323b5b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:55:37.706422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7ebfc-03be-49c8-ae58-8e1632323b5b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:37.707790 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d7ebfc-03be-49c8-ae58-8e1632323b5b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:37.717689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 599/1189] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:55:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:55:37.723398 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cdfb5-f937-460b-ae8d-3095c9e096b6 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:37.827078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9cdfb5-f937-460b-ae8d-3095c9e096b6 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:37.860624 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ff7e7-e641-4fff-ace3-d014b09cbe44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:37.861523 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ff7e7-e641-4fff-ace3-d014b09cbe44 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:37.861807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ff7e7-e641-4fff-ace3-d014b09cbe44 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:37.862083 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-163ff7e7-e641-4fff-ace3-d014b09cbe44 None admin] PATH_INFO: `/v3/users/416de5d8cfb04f7a822a99268a89c959` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:37.863381 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ff7e7-e641-4fff-ace3-d014b09cbe44 None admin] RBAC: Authorizing `identity:get_user(user_id=416de5d8cfb04f7a822a99268a89c959)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:37.864879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-163ff7e7-e641-4fff-ace3-d014b09cbe44 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:37.866918 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 591/1190] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:55:37 2022] GET /identity/v3/users/416de5d8cfb04f7a822a99268a89c959 => generated 311 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:55:37.872830 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7ebfc-03be-49c8-ae58-8e1632323b5b None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:37.958950 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:37.971436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d7ebfc-03be-49c8-ae58-8e1632323b5b None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:38.003959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a3bae-e7d3-4076-8e99-001ca32dc668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:38.004831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a3bae-e7d3-4076-8e99-001ca32dc668 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:38.005118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a3bae-e7d3-4076-8e99-001ca32dc668 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:38.005398 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a3bae-e7d3-4076-8e99-001ca32dc668 None admin] PATH_INFO: `/v3/projects/e0eebdae09464776bfd0e92e5c017e3e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:38.006607 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a3bae-e7d3-4076-8e99-001ca32dc668 None admin] RBAC: Authorizing `identity:get_project(project_id=e0eebdae09464776bfd0e92e5c017e3e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:38.008038 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a3bae-e7d3-4076-8e99-001ca32dc668 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:38.009765 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 600/1191] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:55:37 2022] GET /identity/v3/projects/e0eebdae09464776bfd0e92e5c017e3e => generated 361 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:38.016625 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ff7e7-e641-4fff-ace3-d014b09cbe44 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:38.020443 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:38.115708 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-163ff7e7-e641-4fff-ace3-d014b09cbe44 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:38.149015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a10ba888-e55c-4145-899d-54516f32e5c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:38.149883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ba888-e55c-4145-899d-54516f32e5c6 None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:38.150162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ba888-e55c-4145-899d-54516f32e5c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:38.150523 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a10ba888-e55c-4145-899d-54516f32e5c6 None admin] PATH_INFO: `/v3/projects/e0eebdae09464776bfd0e92e5c017e3e/users/416de5d8cfb04f7a822a99268a89c959/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:38.153081 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ba888-e55c-4145-899d-54516f32e5c6 None admin] RBAC: Authorizing `identity:create_grant(project_id=e0eebdae09464776bfd0e92e5c017e3e, user_id=416de5d8cfb04f7a822a99268a89c959, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:38.154516 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a10ba888-e55c-4145-899d-54516f32e5c6 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:38.167986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 592/1192] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:55:38 2022] PUT /identity/v3/projects/e0eebdae09464776bfd0e92e5c017e3e/users/416de5d8cfb04f7a822a99268a89c959/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:55:40.182207 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8843b-c8d6-4972-8967-84644fc9216f None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:40.182800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8843b-c8d6-4972-8967-84644fc9216f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:40.183446 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb8843b-c8d6-4972-8967-84644fc9216f None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:40.184760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 601/1193] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:55:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:55:40.189957 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb1d3b-4d8e-460c-b387-bab7d8d489e8 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:40.190405 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb1d3b-4d8e-460c-b387-bab7d8d489e8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:40.190546 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4efb1d3b-4d8e-460c-b387-bab7d8d489e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:40.207666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4efb1d3b-4d8e-460c-b387-bab7d8d489e8 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:55:40.208853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4efb1d3b-4d8e-460c-b387-bab7d8d489e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:55:40.266712 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:40.276526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 593/1194] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:55:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:55:40.300824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8843b-c8d6-4972-8967-84644fc9216f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:40.402034 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb8843b-c8d6-4972-8967-84644fc9216f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:40.435173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcc7f1-5450-46a6-a749-cf8190982c97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:40.435948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcc7f1-5450-46a6-a749-cf8190982c97 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:40.436228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcc7f1-5450-46a6-a749-cf8190982c97 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:40.436502 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5adcc7f1-5450-46a6-a749-cf8190982c97 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:40.446424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5adcc7f1-5450-46a6-a749-cf8190982c97 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:55:40.453969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-5adcc7f1-5450-46a6-a749-cf8190982c97 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:55:40.453969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:55:40.453969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:55:40.453969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:55:40.453969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:55:40.453969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:55:40.453969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:55:40.453969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:55:40.453969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:55:40.453969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:55:40.453969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:55:40.453969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:55:40.453969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:55:40.453969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:55:40.453969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:55:40.453969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:55:40.455140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:55:40.455140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:55:40.455140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:55:40.455140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:55:40.455140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:55:40.455140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:55:40.455140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:55:40.455140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:55:40.455140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:55:40.455140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:55:40.455140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:55:40.455140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:55:40.455140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:55:40.455140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:55:40.455140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:55:40.455140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:55:40.456070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:55:40.456070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:55:40.456070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:55:40.456070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:55:40.456070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:55:40.456070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:55:40.456070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:55:40.456070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:55:40.456070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:55:40.456070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:55:40.456070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:55:40.456070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:55:40.456070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 602/1195] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:55:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:55:40.461332 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb1d3b-4d8e-460c-b387-bab7d8d489e8 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:40.558097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4efb1d3b-4d8e-460c-b387-bab7d8d489e8 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:40.591765 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3a5c5-9e69-4bfd-8187-c5d3506075a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:40.592425 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3a5c5-9e69-4bfd-8187-c5d3506075a5 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:40.592712 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3a5c5-9e69-4bfd-8187-c5d3506075a5 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:40.592986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e3a5c5-9e69-4bfd-8187-c5d3506075a5 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:40.593725 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3a5c5-9e69-4bfd-8187-c5d3506075a5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:55:40.594031 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3a5c5-9e69-4bfd-8187-c5d3506075a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:40.595418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e3a5c5-9e69-4bfd-8187-c5d3506075a5 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:40.603322 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 594/1196] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:55:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:40.609679 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcc7f1-5450-46a6-a749-cf8190982c97 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:40.628227 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:40.708398 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5adcc7f1-5450-46a6-a749-cf8190982c97 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:40.742163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abfd295-6d26-4552-901a-86d6ded1dd86 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:40.743067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfd295-6d26-4552-901a-86d6ded1dd86 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:40.743355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfd295-6d26-4552-901a-86d6ded1dd86 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:40.743632 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abfd295-6d26-4552-901a-86d6ded1dd86 None admin] PATH_INFO: `/v3/users/416de5d8cfb04f7a822a99268a89c959` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:40.744837 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abfd295-6d26-4552-901a-86d6ded1dd86 None admin] RBAC: Authorizing `identity:get_user(user_id=416de5d8cfb04f7a822a99268a89c959)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:40.746313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abfd295-6d26-4552-901a-86d6ded1dd86 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:40.747981 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 603/1197] 10.222.0.49 () {56 vars in 1177 bytes} [Mon Jul 11 13:55:40 2022] GET /identity/v3/users/416de5d8cfb04f7a822a99268a89c959 => generated 311 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:40.754357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3a5c5-9e69-4bfd-8187-c5d3506075a5 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:40.816691 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:40.860662 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e3a5c5-9e69-4bfd-8187-c5d3506075a5 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:40.899580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fff2a2-baf3-4af5-b193-449c5dd4047d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:40.900234 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fff2a2-baf3-4af5-b193-449c5dd4047d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:40.900519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fff2a2-baf3-4af5-b193-449c5dd4047d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:40.900791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02fff2a2-baf3-4af5-b193-449c5dd4047d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:40.968164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fff2a2-baf3-4af5-b193-449c5dd4047d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:40.969540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02fff2a2-baf3-4af5-b193-449c5dd4047d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:41.023368 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:41.069080 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 595/1198] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:55:40 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 13:55:41.075524 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abfd295-6d26-4552-901a-86d6ded1dd86 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:41.166549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:41.184889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abfd295-6d26-4552-901a-86d6ded1dd86 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:41.221380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7038906b-e77f-46d0-bef3-05fc21ae0edf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:41.222160 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038906b-e77f-46d0-bef3-05fc21ae0edf None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:41.222487 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038906b-e77f-46d0-bef3-05fc21ae0edf None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:41.222764 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7038906b-e77f-46d0-bef3-05fc21ae0edf None admin] PATH_INFO: `/v3/projects/e0eebdae09464776bfd0e92e5c017e3e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:41.223970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038906b-e77f-46d0-bef3-05fc21ae0edf None admin] RBAC: Authorizing `identity:get_project(project_id=e0eebdae09464776bfd0e92e5c017e3e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:41.225398 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7038906b-e77f-46d0-bef3-05fc21ae0edf None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:41.227192 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 604/1199] 10.222.0.49 () {56 vars in 1186 bytes} [Mon Jul 11 13:55:41 2022] GET /identity/v3/projects/e0eebdae09464776bfd0e92e5c017e3e => generated 361 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:41.233800 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fff2a2-baf3-4af5-b193-449c5dd4047d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:55:41.339791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02fff2a2-baf3-4af5-b193-449c5dd4047d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:55:41.374735 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8939290-c576-429d-994a-7036e2ca59a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:55:41.375385 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8939290-c576-429d-994a-7036e2ca59a2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:55:41.375678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8939290-c576-429d-994a-7036e2ca59a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:55:41.375960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8939290-c576-429d-994a-7036e2ca59a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:55:41.376793 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8939290-c576-429d-994a-7036e2ca59a2 None admin] RBAC: Adding query filter params (role.id=6964639a7f6c41e8a6701ef533151c41, scope.project.id=e0eebdae09464776bfd0e92e5c017e3e, user.id=416de5d8cfb04f7a822a99268a89c959) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:55:41.377146 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8939290-c576-429d-994a-7036e2ca59a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:55:41.378706 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8939290-c576-429d-994a-7036e2ca59a2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:55:41.384555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 596/1200] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:55:41 2022] GET /identity/v3/role_assignments?user.id=416de5d8cfb04f7a822a99268a89c959&scope.project.id=e0eebdae09464776bfd0e92e5c017e3e&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:55:51.216706 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=24 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:55:51.381922 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=90 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:37.593884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424fade-1534-4d4e-a34a-8da1edba6842 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:37.593884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424fade-1534-4d4e-a34a-8da1edba6842 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:37.593884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3424fade-1534-4d4e-a34a-8da1edba6842 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:37.593884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 605/1201] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:56:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:56:37.599919 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105b827-6b30-445a-84af-3fff9a98a447 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:37.600450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105b827-6b30-445a-84af-3fff9a98a447 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:37.600837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7105b827-6b30-445a-84af-3fff9a98a447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:37.620652 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-7105b827-6b30-445a-84af-3fff9a98a447 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:56:37.621872 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7105b827-6b30-445a-84af-3fff9a98a447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:56:37.708494 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 597/1202] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:56:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:56:37.735481 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424fade-1534-4d4e-a34a-8da1edba6842 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:37.882842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3424fade-1534-4d4e-a34a-8da1edba6842 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:37.940736 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537a688-d336-4491-9215-afc64997e7d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:37.941558 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537a688-d336-4491-9215-afc64997e7d8 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:37.941848 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537a688-d336-4491-9215-afc64997e7d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:37.942127 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1537a688-d336-4491-9215-afc64997e7d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:37.961743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537a688-d336-4491-9215-afc64997e7d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:37.963459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1537a688-d336-4491-9215-afc64997e7d8 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:37.984131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-1537a688-d336-4491-9215-afc64997e7d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:56:37.984131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:37.984131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:37.984131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:37.984131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:37.984131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:37.984131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:37.984131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:37.984131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:37.984131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:37.984131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:37.984131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:37.984131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:56:37.984131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:56:37.984131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:56:37.985579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:56:37.985579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:37.985579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:37.985579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:37.985579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:37.985579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:37.985579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:37.985579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:37.985579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:37.985579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:37.985579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:37.985579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:37.985579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:37.985579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:37.985579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:37.985579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:37.985579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:37.986601 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:56:37.986601 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:56:37.986601 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:56:37.986601 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:56:37.986601 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 606/1203] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:56:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:56:37.994081 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105b827-6b30-445a-84af-3fff9a98a447 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:38.132927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7105b827-6b30-445a-84af-3fff9a98a447 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:38.169048 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:38.176672 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d0d0bf-92d1-4bec-b462-e93062c9f626 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:38.177517 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0d0bf-92d1-4bec-b462-e93062c9f626 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:38.177853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0d0bf-92d1-4bec-b462-e93062c9f626 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:38.178135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0d0bf-92d1-4bec-b462-e93062c9f626 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:38.179058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d0d0bf-92d1-4bec-b462-e93062c9f626 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:38.179676 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d0d0bf-92d1-4bec-b462-e93062c9f626 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:38.181244 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d0d0bf-92d1-4bec-b462-e93062c9f626 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:38.195516 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 598/1204] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:56:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:38.202183 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537a688-d336-4491-9215-afc64997e7d8 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:38.271284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:38.340716 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1537a688-d336-4491-9215-afc64997e7d8 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:38.388712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfea7999-2c93-435e-8e88-d3033b41bede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:38.389438 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea7999-2c93-435e-8e88-d3033b41bede None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:38.389712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea7999-2c93-435e-8e88-d3033b41bede None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:38.389991 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfea7999-2c93-435e-8e88-d3033b41bede None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:38.390808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea7999-2c93-435e-8e88-d3033b41bede None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:38.392379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfea7999-2c93-435e-8e88-d3033b41bede None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:38.420249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 607/1205] 10.222.0.49 () {60 vars in 1132 bytes} [Mon Jul 11 13:56:38 2022] POST /identity/v3/users => generated 251 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:56:40.764560 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22746e-2533-4cc4-b26f-33d667b57d09 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:40.764560 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22746e-2533-4cc4-b26f-33d667b57d09 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:40.764560 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f22746e-2533-4cc4-b26f-33d667b57d09 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:40.765499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 599/1206] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:56:40 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:56:40.771070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21856133-c95a-4b3d-a43a-1a52ec121de0 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:40.771624 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21856133-c95a-4b3d-a43a-1a52ec121de0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:40.771624 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21856133-c95a-4b3d-a43a-1a52ec121de0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:40.789734 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-21856133-c95a-4b3d-a43a-1a52ec121de0 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:56:40.790986 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-21856133-c95a-4b3d-a43a-1a52ec121de0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:56:40.863288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 608/1207] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:56:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:56:40.887889 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22746e-2533-4cc4-b26f-33d667b57d09 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:41.019235 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f22746e-2533-4cc4-b26f-33d667b57d09 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:41.069026 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0483c-64b5-4dc3-b8ab-344b42b98612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:41.070251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0483c-64b5-4dc3-b8ab-344b42b98612 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:41.070733 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0483c-64b5-4dc3-b8ab-344b42b98612 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:41.071020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6eb0483c-64b5-4dc3-b8ab-344b42b98612 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:41.087088 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6eb0483c-64b5-4dc3-b8ab-344b42b98612 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:56:41.096456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-6eb0483c-64b5-4dc3-b8ab-344b42b98612 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:56:41.096456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:41.096456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:41.096456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:41.096456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:41.096456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:41.096456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:41.096456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:41.096456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:41.096456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:41.096456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:41.096456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:41.096456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:56:41.096456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:56:41.096456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:56:41.096456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:56:41.097441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:56:41.097441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:56:41.097441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:56:41.097441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:56:41.097441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:41.097441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:41.097441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:41.097441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:41.097441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:41.097441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:41.097441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:41.097441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:41.097441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:41.097441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:41.097441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:41.097441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:41.098450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:41.098450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:41.098450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:41.098450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:41.098450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:56:41.098450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:56:41.098450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:56:41.098450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:56:41.098450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:56:41.098450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:56:41.098450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:56:41.098450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:56:41.098450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 600/1208] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:56:40 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:56:41.104096 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21856133-c95a-4b3d-a43a-1a52ec121de0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:41.152586 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:41.226937 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21856133-c95a-4b3d-a43a-1a52ec121de0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:41.285080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a2962-f44b-4c3e-90b5-15c2f9affcc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:41.285080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a2962-f44b-4c3e-90b5-15c2f9affcc0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:41.285370 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a2962-f44b-4c3e-90b5-15c2f9affcc0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:41.285629 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe2a2962-f44b-4c3e-90b5-15c2f9affcc0 None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:41.286347 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a2962-f44b-4c3e-90b5-15c2f9affcc0 None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:41.286741 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a2962-f44b-4c3e-90b5-15c2f9affcc0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:41.288463 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe2a2962-f44b-4c3e-90b5-15c2f9affcc0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:41.298995 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 609/1209] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:56:41 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 13:56:41.305577 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0483c-64b5-4dc3-b8ab-344b42b98612 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:41.370504 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:41.441171 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6eb0483c-64b5-4dc3-b8ab-344b42b98612 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:41.481490 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84844ed8-66f5-4774-86c7-6ceed63b4057 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:41.482294 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84844ed8-66f5-4774-86c7-6ceed63b4057 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:41.482600 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84844ed8-66f5-4774-86c7-6ceed63b4057 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:41.482874 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84844ed8-66f5-4774-86c7-6ceed63b4057 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:41.496694 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84844ed8-66f5-4774-86c7-6ceed63b4057 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:41.498338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84844ed8-66f5-4774-86c7-6ceed63b4057 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:41.518438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-84844ed8-66f5-4774-86c7-6ceed63b4057 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:56:41.518438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:41.518438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:41.518438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:41.518438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:41.518438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:41.518438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:41.518438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:41.518438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:41.518438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:41.518438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:41.518438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:41.518438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:56:41.518438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:56:41.518438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:56:41.519792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:56:41.519792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:41.519792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:41.519792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:41.519792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:41.519792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:41.519792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:41.519792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:41.519792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:41.519792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:41.519792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:41.519792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:41.519792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:41.519792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:41.519792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:41.519792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:41.519792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:41.520915 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:56:41.520915 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:56:41.520915 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:56:41.520915 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:56:41.520915 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 601/1210] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:56:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:56:41.526163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a2962-f44b-4c3e-90b5-15c2f9affcc0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:41.664604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:41.669175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe2a2962-f44b-4c3e-90b5-15c2f9affcc0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:41.715495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3b3e4-2845-4882-92bb-84fef54d83c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:41.716302 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3b3e4-2845-4882-92bb-84fef54d83c0 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:41.716588 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3b3e4-2845-4882-92bb-84fef54d83c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:41.716917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a3b3e4-2845-4882-92bb-84fef54d83c0 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:41.717866 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a3b3e4-2845-4882-92bb-84fef54d83c0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:41.718297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a3b3e4-2845-4882-92bb-84fef54d83c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:41.719893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a3b3e4-2845-4882-92bb-84fef54d83c0 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:41.739119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 610/1211] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:56:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:41.739337 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84844ed8-66f5-4774-86c7-6ceed63b4057 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:41.859148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84844ed8-66f5-4774-86c7-6ceed63b4057 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:41.860814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:41.916876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed02ad-ec9b-4e64-818a-32bff95f9a56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:41.917750 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed02ad-ec9b-4e64-818a-32bff95f9a56 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:41.918150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed02ad-ec9b-4e64-818a-32bff95f9a56 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:41.918517 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aed02ad-ec9b-4e64-818a-32bff95f9a56 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:41.931861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed02ad-ec9b-4e64-818a-32bff95f9a56 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:41.933722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aed02ad-ec9b-4e64-818a-32bff95f9a56 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:41.953468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-8aed02ad-ec9b-4e64-818a-32bff95f9a56 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 13:56:41.953468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:41.953468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:41.953468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:41.953468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:41.953468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:41.953468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:41.953468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:41.953468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:41.953468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:41.953468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:41.953468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:41.953468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:56:41.953468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:56:41.953468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:56:41.953468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:56:41.955485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:41.955485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:41.955485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:56:41.955485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:56:41.955485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:56:41.955485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:56:41.955485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:41.955485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:41.955485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:41.955485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:41.955485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:41.955485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:41.955485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:41.955485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:41.955485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:41.955485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:41.955485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:41.955485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:41.956540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:41.956540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:41.956540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:56:41.956540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:56:41.956540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:56:41.956540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:56:41.956540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:56:41.956540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:56:41.956540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 13:56:41.956540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:56:41.956540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 602/1212] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:56:41 2022] GET /identity/v3/users/placement => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:56:41.962336 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3b3e4-2845-4882-92bb-84fef54d83c0 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:42.090971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a3b3e4-2845-4882-92bb-84fef54d83c0 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:42.134168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a3e7b-86f7-4f2e-b0fb-4c08f81ba429 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:42.135185 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a3e7b-86f7-4f2e-b0fb-4c08f81ba429 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:42.135711 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a3e7b-86f7-4f2e-b0fb-4c08f81ba429 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:42.136765 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421a3e7b-86f7-4f2e-b0fb-4c08f81ba429 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:42.138003 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a3e7b-86f7-4f2e-b0fb-4c08f81ba429 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:42.138521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a3e7b-86f7-4f2e-b0fb-4c08f81ba429 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:42.140311 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421a3e7b-86f7-4f2e-b0fb-4c08f81ba429 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:42.169137 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 611/1213] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Jul 11 13:56:41 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:42.170524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed02ad-ec9b-4e64-818a-32bff95f9a56 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:42.307724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aed02ad-ec9b-4e64-818a-32bff95f9a56 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:42.351188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c8353-7d2d-4157-b5ce-83689c1ce25b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:42.351844 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c8353-7d2d-4157-b5ce-83689c1ce25b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:42.352133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c8353-7d2d-4157-b5ce-83689c1ce25b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:42.352410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69c8353-7d2d-4157-b5ce-83689c1ce25b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:42.389275 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:42.436545 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c8353-7d2d-4157-b5ce-83689c1ce25b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:42.438097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69c8353-7d2d-4157-b5ce-83689c1ce25b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:42.561689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 603/1214] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:56:42 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 11 13:56:42.567444 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a3e7b-86f7-4f2e-b0fb-4c08f81ba429 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:42.588351 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:42.693742 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421a3e7b-86f7-4f2e-b0fb-4c08f81ba429 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:42.740828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de46be-e7ba-4811-8467-78b5b2e6bf9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:42.741789 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de46be-e7ba-4811-8467-78b5b2e6bf9f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:42.742324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de46be-e7ba-4811-8467-78b5b2e6bf9f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:42.742714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36de46be-e7ba-4811-8467-78b5b2e6bf9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:42.754592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de46be-e7ba-4811-8467-78b5b2e6bf9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:42.756210 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36de46be-e7ba-4811-8467-78b5b2e6bf9f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:42.776163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-36de46be-e7ba-4811-8467-78b5b2e6bf9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:56:42.776163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:42.776163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:42.776163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:42.776163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:42.776163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:42.776163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:42.776163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:42.776163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:42.776163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:42.776163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:42.776163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:42.776163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:56:42.776163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:56:42.776163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:56:42.777643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:56:42.777643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:42.777643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:42.777643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:42.777643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:42.777643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:42.777643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:42.777643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:42.777643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:42.777643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:42.777643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:42.777643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:42.777643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:42.777643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:42.777643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:42.777643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:42.777643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:42.778816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:56:42.778816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:56:42.778816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:56:42.778816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:56:42.785442 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 612/1215] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:56:42 2022] GET /identity/v3/domains/Default => generated 87 bytes in 213 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:56:42.786989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c8353-7d2d-4157-b5ce-83689c1ce25b None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:42.859169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:42.901706 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69c8353-7d2d-4157-b5ce-83689c1ce25b None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:42.941631 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b384cf9-293a-4321-b8ee-6f40fce0570a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:42.942529 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b384cf9-293a-4321-b8ee-6f40fce0570a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:42.942920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b384cf9-293a-4321-b8ee-6f40fce0570a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:42.943349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b384cf9-293a-4321-b8ee-6f40fce0570a None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:42.944266 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b384cf9-293a-4321-b8ee-6f40fce0570a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:42.944641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b384cf9-293a-4321-b8ee-6f40fce0570a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:42.946187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b384cf9-293a-4321-b8ee-6f40fce0570a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:42.965187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 604/1216] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:42.966479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de46be-e7ba-4811-8467-78b5b2e6bf9f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:43.053044 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:43.097836 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36de46be-e7ba-4811-8467-78b5b2e6bf9f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:43.159731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7cc5ff-5e07-4c0e-a399-e3f62a795692 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:43.160957 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cc5ff-5e07-4c0e-a399-e3f62a795692 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:43.161384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cc5ff-5e07-4c0e-a399-e3f62a795692 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:43.161816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7cc5ff-5e07-4c0e-a399-e3f62a795692 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:43.174584 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7cc5ff-5e07-4c0e-a399-e3f62a795692 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:43.177282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7cc5ff-5e07-4c0e-a399-e3f62a795692 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:43.204535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-7a7cc5ff-5e07-4c0e-a399-e3f62a795692 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:56:43.204535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:43.204535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:43.204535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:43.204535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:43.204535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:43.204535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:43.204535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:43.204535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:43.204535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:43.204535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:43.204535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:43.204535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:56:43.204535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:56:43.204535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:56:43.205643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:56:43.205643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:43.205643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:43.205643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:43.205643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:43.205643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:43.205643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:43.205643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:43.205643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:43.205643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:43.205643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:43.205643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:43.205643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:43.205643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:43.205643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:43.205643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:43.205643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:43.206695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:56:43.206695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:56:43.206695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:56:43.206695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:56:43.206695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:56:43.206695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:56:43.206695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:56:43.206695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:56:43.215515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 613/1217] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:56:42 2022] GET /identity/v3/projects/service => generated 88 bytes in 248 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:56:43.215735 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b384cf9-293a-4321-b8ee-6f40fce0570a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:43.351438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b384cf9-293a-4321-b8ee-6f40fce0570a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:43.382591 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:43.397571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca59ea53-4987-4bea-ab9d-8979211084e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:43.398365 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59ea53-4987-4bea-ab9d-8979211084e8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:43.398726 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59ea53-4987-4bea-ab9d-8979211084e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:43.399074 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca59ea53-4987-4bea-ab9d-8979211084e8 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:43.399973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59ea53-4987-4bea-ab9d-8979211084e8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:43.400345 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59ea53-4987-4bea-ab9d-8979211084e8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:43.402403 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca59ea53-4987-4bea-ab9d-8979211084e8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:43.423026 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 605/1218] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:56:43 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:43.425117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7cc5ff-5e07-4c0e-a399-e3f62a795692 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:43.538652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7cc5ff-5e07-4c0e-a399-e3f62a795692 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:43.574011 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:43.580346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e326eeb6-9422-4f88-a190-2f4371dc015f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:43.581281 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326eeb6-9422-4f88-a190-2f4371dc015f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:43.581630 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326eeb6-9422-4f88-a190-2f4371dc015f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:43.581956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e326eeb6-9422-4f88-a190-2f4371dc015f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:43.582905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326eeb6-9422-4f88-a190-2f4371dc015f None admin] RBAC: Adding query filter params (role.id=1f7cf26664d04e5e9331ead293476442, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=ee29b7f1cea64d6781e1df5c8021363f) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:43.583275 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326eeb6-9422-4f88-a190-2f4371dc015f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:43.584776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e326eeb6-9422-4f88-a190-2f4371dc015f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:43.598726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 614/1219] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:56:43 2022] GET /identity/v3/role_assignments?user.id=ee29b7f1cea64d6781e1df5c8021363f&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=1f7cf26664d04e5e9331ead293476442 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:45.637026 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9624cd-a5cf-41e1-b925-122ffb0f286f None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:45.637609 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9624cd-a5cf-41e1-b925-122ffb0f286f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:45.637723 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9624cd-a5cf-41e1-b925-122ffb0f286f None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:45.639341 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 606/1220] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:56:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:56:45.644794 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6f90a-0d9b-42b9-9af6-ac07b0f702d3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:45.645291 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6f90a-0d9b-42b9-9af6-ac07b0f702d3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:45.645537 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8e6f90a-0d9b-42b9-9af6-ac07b0f702d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:45.662978 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c8e6f90a-0d9b-42b9-9af6-ac07b0f702d3 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:56:45.664145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c8e6f90a-0d9b-42b9-9af6-ac07b0f702d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:56:45.738222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 615/1221] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:56:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:56:45.763301 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9624cd-a5cf-41e1-b925-122ffb0f286f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:45.889556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9624cd-a5cf-41e1-b925-122ffb0f286f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:45.937634 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3e3c3-0876-4394-987a-cc38b31e652f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:45.938478 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3e3c3-0876-4394-987a-cc38b31e652f None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:45.938774 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3e3c3-0876-4394-987a-cc38b31e652f None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:45.939053 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00f3e3c3-0876-4394-987a-cc38b31e652f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:45.950534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00f3e3c3-0876-4394-987a-cc38b31e652f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:56:45.958424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-00f3e3c3-0876-4394-987a-cc38b31e652f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:56:45.958424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:45.958424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:45.958424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:45.958424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:45.958424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:45.958424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:45.958424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:45.958424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:45.958424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:45.958424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:45.958424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:45.958424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:56:45.958424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:56:45.958424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:56:45.958424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:56:45.959607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:56:45.959607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:56:45.959607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:56:45.959607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:56:45.959607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:45.959607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:45.959607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:45.959607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:45.959607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:45.959607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:45.959607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:45.959607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:45.959607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:45.959607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:45.959607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:45.959607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:45.966042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:45.966042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:45.966042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:45.966042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:45.966042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:56:45.966042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:56:45.966042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:56:45.966042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:56:45.966042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:56:45.966042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:56:45.966042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:56:45.966042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:56:45.966042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 607/1222] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:56:45 2022] GET /identity/v3/roles/service => generated 85 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:56:45.966824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6f90a-0d9b-42b9-9af6-ac07b0f702d3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:46.077230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8e6f90a-0d9b-42b9-9af6-ac07b0f702d3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:46.107327 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:46.121926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb71cdf-e813-4a7a-a481-17326b67456f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:46.122751 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb71cdf-e813-4a7a-a481-17326b67456f None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:46.123163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb71cdf-e813-4a7a-a481-17326b67456f None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:46.123403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb71cdf-e813-4a7a-a481-17326b67456f None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:46.124536 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb71cdf-e813-4a7a-a481-17326b67456f None admin] RBAC: Adding query filter params (name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:46.124802 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb71cdf-e813-4a7a-a481-17326b67456f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:46.126115 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb71cdf-e813-4a7a-a481-17326b67456f None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:46.136657 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 616/1223] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:56:45 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:46.142378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3e3c3-0876-4394-987a-cc38b31e652f None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:46.188945 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:46.256579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00f3e3c3-0876-4394-987a-cc38b31e652f None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:46.298312 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dec6ca-eddc-4d45-bfe8-5c5de0aedf34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:46.299164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dec6ca-eddc-4d45-bfe8-5c5de0aedf34 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:46.299477 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dec6ca-eddc-4d45-bfe8-5c5de0aedf34 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:46.299784 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9dec6ca-eddc-4d45-bfe8-5c5de0aedf34 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:46.313961 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dec6ca-eddc-4d45-bfe8-5c5de0aedf34 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:46.317238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9dec6ca-eddc-4d45-bfe8-5c5de0aedf34 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:46.336542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-f9dec6ca-eddc-4d45-bfe8-5c5de0aedf34 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:56:46.336542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:46.336542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:46.336542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:46.336542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:46.336542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:46.336542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:46.336542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:46.336542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:46.336542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:46.336542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:46.336542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:46.336542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:56:46.336542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:56:46.336542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:56:46.337794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:56:46.337794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:46.337794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:46.337794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:46.337794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:46.337794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:46.337794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:46.337794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:46.337794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:46.337794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:46.337794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:46.337794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:46.337794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:46.337794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:46.337794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:46.337794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:46.337794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:46.338929 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:56:46.338929 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:56:46.338929 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:56:46.338929 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:56:46.338929 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 608/1224] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:56:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:56:46.344303 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb71cdf-e813-4a7a-a481-17326b67456f None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:46.469068 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb71cdf-e813-4a7a-a481-17326b67456f None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:46.508444 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b3e8c-a77f-4f48-b6fc-5401825bf26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:46.509123 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b3e8c-a77f-4f48-b6fc-5401825bf26d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:46.509418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b3e8c-a77f-4f48-b6fc-5401825bf26d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:46.509755 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d1b3e8c-a77f-4f48-b6fc-5401825bf26d None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:46.510488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b3e8c-a77f-4f48-b6fc-5401825bf26d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:46.510816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b3e8c-a77f-4f48-b6fc-5401825bf26d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:46.512213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d1b3e8c-a77f-4f48-b6fc-5401825bf26d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:46.526173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 617/1225] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:56:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:46.532671 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dec6ca-eddc-4d45-bfe8-5c5de0aedf34 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:46.640367 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:46.653000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9dec6ca-eddc-4d45-bfe8-5c5de0aedf34 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:46.691751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498728b-764d-482e-9606-489c2fbd226c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:46.692566 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498728b-764d-482e-9606-489c2fbd226c None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:46.692846 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498728b-764d-482e-9606-489c2fbd226c None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:46.693112 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7498728b-764d-482e-9606-489c2fbd226c None admin] PATH_INFO: `/v3/users/placement` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:46.701189 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498728b-764d-482e-9606-489c2fbd226c None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:46.702666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7498728b-764d-482e-9606-489c2fbd226c None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:46.718418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-7498728b-764d-482e-9606-489c2fbd226c None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 13:56:46.718418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:46.718418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:46.718418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:46.718418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:46.718418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:46.718418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:46.718418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:46.718418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:46.718418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:46.718418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:46.718418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:46.718418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:56:46.718418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:56:46.718418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:56:46.718418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:56:46.724949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:46.724949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:46.724949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:56:46.724949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:56:46.724949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:56:46.724949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:56:46.724949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:46.724949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:46.724949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:46.724949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:46.724949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:46.724949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:46.724949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:46.724949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:46.724949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:46.724949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:46.724949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:46.724949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:46.726205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:46.726205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:46.726205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:56:46.726205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:56:46.726205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:56:46.726205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:56:46.726205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:56:46.726205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:56:46.726205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 13:56:46.726205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:56:46.726205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 609/1226] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:56:46 2022] GET /identity/v3/users/placement => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:56:46.727215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b3e8c-a77f-4f48-b6fc-5401825bf26d None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:46.778013 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:46.845035 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d1b3e8c-a77f-4f48-b6fc-5401825bf26d None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:46.882428 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009f0ed-0afe-4536-84f9-5c1c797d6e95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:46.883531 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009f0ed-0afe-4536-84f9-5c1c797d6e95 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:46.883531 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009f0ed-0afe-4536-84f9-5c1c797d6e95 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:46.883814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f009f0ed-0afe-4536-84f9-5c1c797d6e95 None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:46.884783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009f0ed-0afe-4536-84f9-5c1c797d6e95 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:46.885093 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009f0ed-0afe-4536-84f9-5c1c797d6e95 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:46.887199 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f009f0ed-0afe-4536-84f9-5c1c797d6e95 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:46.913408 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 618/1227] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Jul 11 13:56:46 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:46.915350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498728b-764d-482e-9606-489c2fbd226c None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:47.031256 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7498728b-764d-482e-9606-489c2fbd226c None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:47.071960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415b27d-003d-4450-852f-72ecee2795b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:47.072769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415b27d-003d-4450-852f-72ecee2795b2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:47.073058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415b27d-003d-4450-852f-72ecee2795b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:47.073826 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f415b27d-003d-4450-852f-72ecee2795b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:47.085491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f415b27d-003d-4450-852f-72ecee2795b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:47.087121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f415b27d-003d-4450-852f-72ecee2795b2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:47.110115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-f415b27d-003d-4450-852f-72ecee2795b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:56:47.110115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:47.110115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:47.110115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:47.110115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:47.110115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:47.110115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:47.110115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:47.110115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:47.110115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:47.110115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:47.110115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:47.110115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:56:47.110115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:56:47.110115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:56:47.111302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:56:47.111302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:47.111302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:47.111302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:47.111302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:47.111302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:47.111302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:47.111302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:47.111302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:47.111302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:47.111302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:47.111302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:47.111302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:47.111302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:47.111302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:47.111302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:47.111302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:47.117302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:56:47.117302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:56:47.117302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:56:47.117302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:56:47.117302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 610/1228] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:56:46 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:56:47.117600 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009f0ed-0afe-4536-84f9-5c1c797d6e95 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:47.231874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f009f0ed-0afe-4536-84f9-5c1c797d6e95 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:47.234555 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:47.270607 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f74d2-bc12-48cd-be30-d5eb6cd21378 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:47.271314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f74d2-bc12-48cd-be30-d5eb6cd21378 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:47.271633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f74d2-bc12-48cd-be30-d5eb6cd21378 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:47.271998 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5f74d2-bc12-48cd-be30-d5eb6cd21378 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:47.272858 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f74d2-bc12-48cd-be30-d5eb6cd21378 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:47.273287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f74d2-bc12-48cd-be30-d5eb6cd21378 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:47.274946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5f74d2-bc12-48cd-be30-d5eb6cd21378 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:47.288658 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 619/1229] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:47.295414 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415b27d-003d-4450-852f-72ecee2795b2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:47.307005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:47.415437 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f415b27d-003d-4450-852f-72ecee2795b2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:47.461742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23045d3d-d055-4646-959e-8b1e7fb324e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:47.462612 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23045d3d-d055-4646-959e-8b1e7fb324e0 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:47.462899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23045d3d-d055-4646-959e-8b1e7fb324e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:47.463180 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23045d3d-d055-4646-959e-8b1e7fb324e0 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:47.471813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23045d3d-d055-4646-959e-8b1e7fb324e0 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:47.473498 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23045d3d-d055-4646-959e-8b1e7fb324e0 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:47.491276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-23045d3d-d055-4646-959e-8b1e7fb324e0 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:56:47.491276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:47.491276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:47.491276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:47.491276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:47.491276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:47.491276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:47.491276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:47.491276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:47.491276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:47.491276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:47.491276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:47.491276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:56:47.491276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:56:47.491276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:56:47.492502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:56:47.492502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:47.492502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:47.492502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:47.492502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:47.492502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:47.492502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:47.492502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:47.492502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:47.492502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:47.492502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:47.492502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:47.492502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:47.492502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:47.492502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:47.492502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:47.492502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:47.495428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:56:47.495428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:56:47.495428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:56:47.495428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:56:47.495428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:56:47.495428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:56:47.495428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:56:47.495428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:56:47.495428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 611/1230] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:56:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:56:47.498749 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f74d2-bc12-48cd-be30-d5eb6cd21378 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:47.612294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5f74d2-bc12-48cd-be30-d5eb6cd21378 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:47.650356 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b39f2746-b468-47bf-9e66-8bfa879e65ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:47.651308 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f2746-b468-47bf-9e66-8bfa879e65ba None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:47.652309 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f2746-b468-47bf-9e66-8bfa879e65ba None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:47.652615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b39f2746-b468-47bf-9e66-8bfa879e65ba None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:47.653486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f2746-b468-47bf-9e66-8bfa879e65ba None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:47.653803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f2746-b468-47bf-9e66-8bfa879e65ba None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:47.655364 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b39f2746-b468-47bf-9e66-8bfa879e65ba None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:47.668403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 620/1231] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:56:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:47.674886 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23045d3d-d055-4646-959e-8b1e7fb324e0 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:47.746226 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:47.787463 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23045d3d-d055-4646-959e-8b1e7fb324e0 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:47.827672 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fe2c5f-1b63-4ff2-bb1f-200411d2ab3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:47.828603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe2c5f-1b63-4ff2-bb1f-200411d2ab3a None admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:47.828925 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe2c5f-1b63-4ff2-bb1f-200411d2ab3a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:47.829230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fe2c5f-1b63-4ff2-bb1f-200411d2ab3a None admin] PATH_INFO: `/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/ee29b7f1cea64d6781e1df5c8021363f/roles/1f7cf26664d04e5e9331ead293476442` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:47.847353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe2c5f-1b63-4ff2-bb1f-200411d2ab3a None admin] RBAC: Authorizing `identity:create_grant(project_id=08ee65f8b338491f80e7beac5e316b23, user_id=ee29b7f1cea64d6781e1df5c8021363f, role_id=1f7cf26664d04e5e9331ead293476442)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:47.848968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fe2c5f-1b63-4ff2-bb1f-200411d2ab3a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:47.863486 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 612/1232] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:56:47 2022] PUT /identity/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/ee29b7f1cea64d6781e1df5c8021363f/roles/1f7cf26664d04e5e9331ead293476442 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:56:49.946562 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f60597-40ac-4514-9ccb-b32392ef3934 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:49.947156 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f60597-40ac-4514-9ccb-b32392ef3934 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:49.947156 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f60597-40ac-4514-9ccb-b32392ef3934 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:49.948646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 621/1233] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:56:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:56:49.954588 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b57e7e-3046-435d-97e7-724ebb33481a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:49.955331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b57e7e-3046-435d-97e7-724ebb33481a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:49.955331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b57e7e-3046-435d-97e7-724ebb33481a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:49.972971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-59b57e7e-3046-435d-97e7-724ebb33481a None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:56:49.974238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-59b57e7e-3046-435d-97e7-724ebb33481a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:56:50.050488 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 613/1234] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:56:49 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:56:50.076915 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f60597-40ac-4514-9ccb-b32392ef3934 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:50.109237 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:50.205861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f60597-40ac-4514-9ccb-b32392ef3934 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:50.244663 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754a8df-4173-4677-a2f7-56b5ba9df98c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:50.245512 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754a8df-4173-4677-a2f7-56b5ba9df98c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:50.245796 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754a8df-4173-4677-a2f7-56b5ba9df98c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:50.246075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a754a8df-4173-4677-a2f7-56b5ba9df98c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:50.257825 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a754a8df-4173-4677-a2f7-56b5ba9df98c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:56:50.266477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-a754a8df-4173-4677-a2f7-56b5ba9df98c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:56:50.266477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:50.266477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:50.266477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:50.266477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:50.266477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:50.266477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:50.266477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:50.266477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:50.266477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:50.266477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:50.266477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:50.266477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:56:50.266477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:56:50.266477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:56:50.266477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:56:50.267677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:56:50.267677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:56:50.267677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:56:50.267677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:56:50.267677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:50.267677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:50.267677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:50.267677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:50.267677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:50.267677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:50.267677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:50.267677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:50.267677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:50.267677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:50.267677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:50.267677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:50.268543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:50.268543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:50.268543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:50.268543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:50.268543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:56:50.268543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:56:50.268543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:56:50.268543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:56:50.268543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:56:50.268543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:56:50.268543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 11 13:56:50.268543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:56:50.268543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 622/1235] 10.222.0.49 () {56 vars in 1102 bytes} [Mon Jul 11 13:56:50 2022] GET /identity/v3/roles/service => generated 85 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:56:50.274335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b57e7e-3046-435d-97e7-724ebb33481a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:50.324869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:50.384868 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b57e7e-3046-435d-97e7-724ebb33481a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:50.424284 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c44865-a94c-4bb0-a7d3-c624550d101a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:50.425203 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c44865-a94c-4bb0-a7d3-c624550d101a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:50.425462 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c44865-a94c-4bb0-a7d3-c624550d101a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:50.425795 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c44865-a94c-4bb0-a7d3-c624550d101a None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:50.426722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c44865-a94c-4bb0-a7d3-c624550d101a None admin] RBAC: Adding query filter params (name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:50.427085 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c44865-a94c-4bb0-a7d3-c624550d101a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:50.428667 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c44865-a94c-4bb0-a7d3-c624550d101a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:50.438882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 614/1236] 10.222.0.49 () {56 vars in 1103 bytes} [Mon Jul 11 13:56:50 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:50.445045 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754a8df-4173-4677-a2f7-56b5ba9df98c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:50.552457 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:50.568559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a754a8df-4173-4677-a2f7-56b5ba9df98c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:50.620729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16815efd-3cbb-4684-af98-65b66dbf9801 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:50.621668 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16815efd-3cbb-4684-af98-65b66dbf9801 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:50.621958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16815efd-3cbb-4684-af98-65b66dbf9801 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:50.622269 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16815efd-3cbb-4684-af98-65b66dbf9801 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:50.635930 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16815efd-3cbb-4684-af98-65b66dbf9801 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:50.637390 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16815efd-3cbb-4684-af98-65b66dbf9801 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:50.658287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-16815efd-3cbb-4684-af98-65b66dbf9801 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:56:50.658287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:50.658287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:50.658287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:50.658287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:50.658287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:50.658287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:50.658287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:50.658287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:50.658287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:50.658287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:50.658287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:50.658287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:56:50.658287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:56:50.658287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:56:50.659639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:56:50.659639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:50.659639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:50.659639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:50.659639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:50.659639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:50.659639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:50.659639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:50.659639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:50.659639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:50.659639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:50.659639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:50.659639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:50.659639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:50.659639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:50.659639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:50.659639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:50.660815 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:56:50.660815 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:56:50.660815 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:56:50.660815 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:56:50.660815 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 623/1237] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:56:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 216 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:56:50.666217 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c44865-a94c-4bb0-a7d3-c624550d101a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:50.789160 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c44865-a94c-4bb0-a7d3-c624550d101a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:50.801713 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:50.845863 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984ef15-ea22-4f3c-a823-7b446fc1de59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:50.846572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984ef15-ea22-4f3c-a823-7b446fc1de59 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:50.846934 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984ef15-ea22-4f3c-a823-7b446fc1de59 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:50.847294 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0984ef15-ea22-4f3c-a823-7b446fc1de59 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:50.848124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984ef15-ea22-4f3c-a823-7b446fc1de59 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:50.848516 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984ef15-ea22-4f3c-a823-7b446fc1de59 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:50.850000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0984ef15-ea22-4f3c-a823-7b446fc1de59 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:50.865036 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 615/1238] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:56:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:50.871931 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16815efd-3cbb-4684-af98-65b66dbf9801 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:51.002559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16815efd-3cbb-4684-af98-65b66dbf9801 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:51.049008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9530790-b6a4-4458-9e97-6e06ddffcc62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:51.050764 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9530790-b6a4-4458-9e97-6e06ddffcc62 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:51.051155 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9530790-b6a4-4458-9e97-6e06ddffcc62 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:51.051566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9530790-b6a4-4458-9e97-6e06ddffcc62 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:51.057808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:51.071252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9530790-b6a4-4458-9e97-6e06ddffcc62 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:51.073014 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9530790-b6a4-4458-9e97-6e06ddffcc62 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:51.090729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-d9530790-b6a4-4458-9e97-6e06ddffcc62 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 13:56:51.090729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:51.090729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:51.090729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:51.090729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:51.090729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:51.090729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:51.090729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:51.090729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:51.090729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:51.090729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:51.090729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:51.090729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:56:51.090729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:56:51.090729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:56:51.090729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:56:51.093445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:51.093445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:51.093445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:56:51.093445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:56:51.093445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:56:51.093445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:56:51.093445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:51.093445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:51.093445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:51.093445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:51.093445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:51.093445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:51.093445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:51.093445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:51.093445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:51.093445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:51.093445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:51.093445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:51.095471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:51.095471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:51.095471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:56:51.095471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:56:51.095471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:56:51.095471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:56:51.095471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:56:51.095471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:56:51.095471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 13:56:51.095471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:56:51.095471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 624/1239] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:56:50 2022] GET /identity/v3/users/placement => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:56:51.099300 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984ef15-ea22-4f3c-a823-7b446fc1de59 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:51.238892 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0984ef15-ea22-4f3c-a823-7b446fc1de59 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:51.280814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a385ea4-76c3-436a-b85a-2b6ce9c7e725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:51.281491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a385ea4-76c3-436a-b85a-2b6ce9c7e725 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:51.281776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a385ea4-76c3-436a-b85a-2b6ce9c7e725 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:51.282058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a385ea4-76c3-436a-b85a-2b6ce9c7e725 None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:51.282942 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a385ea4-76c3-436a-b85a-2b6ce9c7e725 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:51.283261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a385ea4-76c3-436a-b85a-2b6ce9c7e725 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:51.284719 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a385ea4-76c3-436a-b85a-2b6ce9c7e725 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:51.305587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 616/1240] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Jul 11 13:56:51 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:51.311646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9530790-b6a4-4458-9e97-6e06ddffcc62 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:51.427607 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9530790-b6a4-4458-9e97-6e06ddffcc62 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:51.466935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1eb97f-ded0-4065-94eb-2a37ddea92c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:51.467608 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1eb97f-ded0-4065-94eb-2a37ddea92c2 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:51.467955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1eb97f-ded0-4065-94eb-2a37ddea92c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:51.468181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f1eb97f-ded0-4065-94eb-2a37ddea92c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:51.544726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:51.552392 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1eb97f-ded0-4065-94eb-2a37ddea92c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:51.554082 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f1eb97f-ded0-4065-94eb-2a37ddea92c2 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:51.676891 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 625/1241] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:56:51 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 13:56:51.683653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a385ea4-76c3-436a-b85a-2b6ce9c7e725 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:51.716099 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:51.800595 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a385ea4-76c3-436a-b85a-2b6ce9c7e725 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:51.843061 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381195c-d654-4e61-9fcb-3b7e0a06807e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:51.843898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381195c-d654-4e61-9fcb-3b7e0a06807e None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:51.844188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381195c-d654-4e61-9fcb-3b7e0a06807e None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:51.844467 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1381195c-d654-4e61-9fcb-3b7e0a06807e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:51.855967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381195c-d654-4e61-9fcb-3b7e0a06807e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:51.857442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1381195c-d654-4e61-9fcb-3b7e0a06807e None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:51.876239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-1381195c-d654-4e61-9fcb-3b7e0a06807e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:56:51.876239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:51.876239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:51.876239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:51.876239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:51.876239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:51.876239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:51.876239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:51.876239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:51.876239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:51.876239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:51.876239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:51.876239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:56:51.876239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:56:51.876239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:56:51.877561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:56:51.877561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:51.877561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:51.877561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:51.877561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:51.877561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:51.877561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:51.877561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:51.877561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:51.877561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:51.877561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:51.877561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:51.877561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:51.877561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:51.877561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:51.877561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:51.877561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:51.878902 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:56:51.878902 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:56:51.878902 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:56:51.878902 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:56:51.878902 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 617/1242] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:56:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:56:51.883867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1eb97f-ded0-4065-94eb-2a37ddea92c2 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:51.989371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:51.995988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f1eb97f-ded0-4065-94eb-2a37ddea92c2 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:52.036247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8154b9-bb85-49dc-8fb0-d3a84a9b5764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:52.036903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8154b9-bb85-49dc-8fb0-d3a84a9b5764 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:52.037195 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8154b9-bb85-49dc-8fb0-d3a84a9b5764 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:52.037473 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f8154b9-bb85-49dc-8fb0-d3a84a9b5764 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:52.038204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8154b9-bb85-49dc-8fb0-d3a84a9b5764 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:52.038581 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8154b9-bb85-49dc-8fb0-d3a84a9b5764 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:52.039973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f8154b9-bb85-49dc-8fb0-d3a84a9b5764 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:52.052710 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 626/1243] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:52.060203 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381195c-d654-4e61-9fcb-3b7e0a06807e None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:52.155770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:52.190552 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1381195c-d654-4e61-9fcb-3b7e0a06807e None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:52.236195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bfe7c-883f-4723-80b0-8350f0e11649 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:52.237363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bfe7c-883f-4723-80b0-8350f0e11649 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:52.237905 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bfe7c-883f-4723-80b0-8350f0e11649 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:52.238426 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-341bfe7c-883f-4723-80b0-8350f0e11649 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:52.247660 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bfe7c-883f-4723-80b0-8350f0e11649 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:52.250909 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-341bfe7c-883f-4723-80b0-8350f0e11649 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:52.268570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-341bfe7c-883f-4723-80b0-8350f0e11649 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:56:52.268570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:52.268570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:52.268570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:52.268570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:52.268570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:52.268570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:52.268570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:52.268570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:52.268570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:52.268570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:52.268570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:52.268570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:56:52.268570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:56:52.268570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:56:52.270628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:56:52.270628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:52.270628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:52.270628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:52.270628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:52.270628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:52.270628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:52.270628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:52.270628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:52.270628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:52.270628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:52.270628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:52.270628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:52.270628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:52.270628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:52.270628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:52.270628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:52.272541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:56:52.272541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:56:52.272541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:56:52.272541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:56:52.272541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:56:52.272541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:56:52.272541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:56:52.272541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:56:52.272541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 618/1244] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:56:52 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:56:52.276647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8154b9-bb85-49dc-8fb0-d3a84a9b5764 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:52.396184 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f8154b9-bb85-49dc-8fb0-d3a84a9b5764 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:52.435256 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547eaaf2-7f39-4a18-baf9-5719e13940bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:52.436030 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547eaaf2-7f39-4a18-baf9-5719e13940bd None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:52.436313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547eaaf2-7f39-4a18-baf9-5719e13940bd None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:52.436592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547eaaf2-7f39-4a18-baf9-5719e13940bd None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:52.437390 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547eaaf2-7f39-4a18-baf9-5719e13940bd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:52.437698 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547eaaf2-7f39-4a18-baf9-5719e13940bd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:52.439144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-547eaaf2-7f39-4a18-baf9-5719e13940bd None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:52.453216 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 627/1245] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:56:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:52.460581 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bfe7c-883f-4723-80b0-8350f0e11649 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:52.577615 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-341bfe7c-883f-4723-80b0-8350f0e11649 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:52.614627 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7710d8c0-7b77-4a22-b632-8afc6f891816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:52.615513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710d8c0-7b77-4a22-b632-8afc6f891816 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:52.615793 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710d8c0-7b77-4a22-b632-8afc6f891816 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:52.616310 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7710d8c0-7b77-4a22-b632-8afc6f891816 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:52.617183 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710d8c0-7b77-4a22-b632-8afc6f891816 None admin] RBAC: Adding query filter params (role.id=1f7cf26664d04e5e9331ead293476442, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=ee29b7f1cea64d6781e1df5c8021363f) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:52.617498 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710d8c0-7b77-4a22-b632-8afc6f891816 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:52.619147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7710d8c0-7b77-4a22-b632-8afc6f891816 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:52.626292 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 619/1246] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:56:52 2022] GET /identity/v3/role_assignments?user.id=ee29b7f1cea64d6781e1df5c8021363f&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=1f7cf26664d04e5e9331ead293476442 => generated 608 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:54.841732 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068c3c3-bf65-430e-80cb-049e13fa6e78 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:54.842351 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068c3c3-bf65-430e-80cb-049e13fa6e78 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:54.842351 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b068c3c3-bf65-430e-80cb-049e13fa6e78 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:54.843905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 628/1247] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:56:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:56:54.850343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea36dc-5579-43d1-ac5e-1f006308aa9c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:54.850862 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea36dc-5579-43d1-ac5e-1f006308aa9c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:54.850862 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0ea36dc-5579-43d1-ac5e-1f006308aa9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:54.857377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:54.873073 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b0ea36dc-5579-43d1-ac5e-1f006308aa9c None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:56:54.874483 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b0ea36dc-5579-43d1-ac5e-1f006308aa9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:56:54.973628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 620/1248] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:56:54 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:56:54.999866 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068c3c3-bf65-430e-80cb-049e13fa6e78 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:55.028866 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:55.149532 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b068c3c3-bf65-430e-80cb-049e13fa6e78 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:55.209357 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087d5de-6ada-49e9-aad7-e50b95ffa3ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:55.210209 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087d5de-6ada-49e9-aad7-e50b95ffa3ec None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:55.210539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087d5de-6ada-49e9-aad7-e50b95ffa3ec None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:55.210826 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c087d5de-6ada-49e9-aad7-e50b95ffa3ec None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:55.222335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c087d5de-6ada-49e9-aad7-e50b95ffa3ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:56:55.230769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-c087d5de-6ada-49e9-aad7-e50b95ffa3ec None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:56:55.230769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:55.230769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:55.230769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:55.230769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:55.230769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:55.230769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:55.230769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:55.230769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:55.230769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:55.230769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:55.230769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:55.230769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:56:55.230769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:56:55.230769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:56:55.230769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:56:55.232158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:56:55.232158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:56:55.232158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:56:55.232158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:56:55.232158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:55.232158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:55.232158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:55.232158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:55.232158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:55.232158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:55.232158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:55.232158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:55.232158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:55.232158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:55.232158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:55.232158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:55.233620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:55.233620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:55.233620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:55.233620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:55.233620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:56:55.233620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:56:55.233620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:56:55.233620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:56:55.233620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:56:55.233620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:56:55.233620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:56:55.233620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:56:55.233620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 629/1249] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:56:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 233 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:56:55.238930 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea36dc-5579-43d1-ac5e-1f006308aa9c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:55.361595 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0ea36dc-5579-43d1-ac5e-1f006308aa9c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:55.398874 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:55.405315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ce06f-41f2-41e7-8ca4-f91d7fa0c9ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:55.406026 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ce06f-41f2-41e7-8ca4-f91d7fa0c9ed None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:55.406506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ce06f-41f2-41e7-8ca4-f91d7fa0c9ed None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:55.406969 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32ce06f-41f2-41e7-8ca4-f91d7fa0c9ed None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:55.407758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32ce06f-41f2-41e7-8ca4-f91d7fa0c9ed None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:55.408069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32ce06f-41f2-41e7-8ca4-f91d7fa0c9ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:55.409487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32ce06f-41f2-41e7-8ca4-f91d7fa0c9ed None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:55.419484 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 621/1250] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:56:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:55.425212 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087d5de-6ada-49e9-aad7-e50b95ffa3ec None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:55.518617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:55.545986 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c087d5de-6ada-49e9-aad7-e50b95ffa3ec None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:55.594100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d841bb2-2f55-4161-a494-ba5041f51994 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:55.595473 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d841bb2-2f55-4161-a494-ba5041f51994 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:55.595798 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d841bb2-2f55-4161-a494-ba5041f51994 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:55.596211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d841bb2-2f55-4161-a494-ba5041f51994 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:55.609207 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d841bb2-2f55-4161-a494-ba5041f51994 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:55.610942 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d841bb2-2f55-4161-a494-ba5041f51994 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:55.631008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-3d841bb2-2f55-4161-a494-ba5041f51994 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:56:55.631008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:55.631008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:55.631008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:55.631008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:55.631008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:55.631008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:55.631008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:55.631008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:55.631008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:55.631008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:55.631008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:55.631008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:56:55.631008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:56:55.631008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:56:55.632282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:56:55.632282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:55.632282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:55.632282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:55.632282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:55.632282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:55.632282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:55.632282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:55.632282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:55.632282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:55.632282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:55.632282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:55.632282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:55.632282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:55.632282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:55.632282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:55.632282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:55.633660 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:56:55.633660 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:56:55.633660 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:56:55.633660 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:56:55.633660 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 630/1251] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:56:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:56:55.638569 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ce06f-41f2-41e7-8ca4-f91d7fa0c9ed None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:55.761023 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32ce06f-41f2-41e7-8ca4-f91d7fa0c9ed None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:55.806387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac56cf8-6068-400c-9756-f182e4bd3b36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:55.807169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac56cf8-6068-400c-9756-f182e4bd3b36 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:55.807510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac56cf8-6068-400c-9756-f182e4bd3b36 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:55.807869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ac56cf8-6068-400c-9756-f182e4bd3b36 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:55.808737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac56cf8-6068-400c-9756-f182e4bd3b36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:55.809070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac56cf8-6068-400c-9756-f182e4bd3b36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:55.810650 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ac56cf8-6068-400c-9756-f182e4bd3b36 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:55.830240 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 622/1252] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:56:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:55.831421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d841bb2-2f55-4161-a494-ba5041f51994 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:55.957282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d841bb2-2f55-4161-a494-ba5041f51994 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:55.991377 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:56.003094 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9e705-4d71-48db-90f2-938660b2d6a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:56.003934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9e705-4d71-48db-90f2-938660b2d6a4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:56.004250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9e705-4d71-48db-90f2-938660b2d6a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:56.004505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30a9e705-4d71-48db-90f2-938660b2d6a4 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:56.013523 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9e705-4d71-48db-90f2-938660b2d6a4 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:56.015005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30a9e705-4d71-48db-90f2-938660b2d6a4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:56.030846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-30a9e705-4d71-48db-90f2-938660b2d6a4 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 13:56:56.030846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:56.030846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:56.030846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:56.030846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:56.030846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:56.030846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:56.030846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:56.030846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:56.030846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:56.030846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:56.030846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:56.030846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:56:56.030846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:56:56.030846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:56:56.030846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:56:56.032179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:56.032179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:56.032179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:56:56.032179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:56:56.032179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:56:56.032179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:56:56.032179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:56.032179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:56.032179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:56.032179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:56.032179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:56.032179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:56.032179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:56.032179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:56.032179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:56.032179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:56.032179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:56.032179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:56.033365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:56.033365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:56.033365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:56:56.033365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:56:56.033365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:56:56.033365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:56:56.033365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:56:56.033365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:56:56.033365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 13:56:56.033365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:56:56.033365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 631/1253] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:56:55 2022] GET /identity/v3/users/placement => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:56:56.037915 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac56cf8-6068-400c-9756-f182e4bd3b36 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:56.111404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:56.164670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ac56cf8-6068-400c-9756-f182e4bd3b36 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:56.205815 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53489f-0d92-4998-a438-c42562bb2e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:56.206813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53489f-0d92-4998-a438-c42562bb2e7a None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:56.207102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53489f-0d92-4998-a438-c42562bb2e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:56.207389 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce53489f-0d92-4998-a438-c42562bb2e7a None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:56.208270 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53489f-0d92-4998-a438-c42562bb2e7a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:56.208734 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53489f-0d92-4998-a438-c42562bb2e7a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:56.210346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce53489f-0d92-4998-a438-c42562bb2e7a None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:56.232899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 623/1254] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Jul 11 13:56:56 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:56.239272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9e705-4d71-48db-90f2-938660b2d6a4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:56.364648 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30a9e705-4d71-48db-90f2-938660b2d6a4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:56.408956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737db76-087a-40a5-a8ec-d1d5eb4cdd27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:56.409621 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737db76-087a-40a5-a8ec-d1d5eb4cdd27 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:56.409910 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737db76-087a-40a5-a8ec-d1d5eb4cdd27 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:56.410198 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b737db76-087a-40a5-a8ec-d1d5eb4cdd27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:56.485513 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:56.502534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737db76-087a-40a5-a8ec-d1d5eb4cdd27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:56.504083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b737db76-087a-40a5-a8ec-d1d5eb4cdd27 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:56.644183 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 632/1255] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:56:56 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 13:56:56.650614 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53489f-0d92-4998-a438-c42562bb2e7a None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:56.769370 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce53489f-0d92-4998-a438-c42562bb2e7a None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:56.796088 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:56.839979 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f540998-5ad0-4da0-8a08-b1e364d55003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:56.840798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f540998-5ad0-4da0-8a08-b1e364d55003 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:56.841085 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f540998-5ad0-4da0-8a08-b1e364d55003 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:56.841367 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f540998-5ad0-4da0-8a08-b1e364d55003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:56.856073 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f540998-5ad0-4da0-8a08-b1e364d55003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:56.857602 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f540998-5ad0-4da0-8a08-b1e364d55003 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:56.877972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-9f540998-5ad0-4da0-8a08-b1e364d55003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:56:56.877972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:56.877972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:56.877972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:56.877972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:56.877972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:56.877972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:56.877972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:56.877972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:56.877972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:56.877972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:56.877972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:56.877972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:56:56.877972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:56:56.877972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:56:56.879231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:56:56.879231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:56.879231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:56.879231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:56.879231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:56.879231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:56.879231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:56.879231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:56.879231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:56.879231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:56.879231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:56.879231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:56.879231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:56.879231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:56.879231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:56.879231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:56.879231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:56.885128 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:56:56.885128 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:56:56.885128 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:56:56.885128 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:56:56.885128 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 624/1256] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:56:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:56:56.886398 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737db76-087a-40a5-a8ec-d1d5eb4cdd27 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:56.990768 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:57.011025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b737db76-087a-40a5-a8ec-d1d5eb4cdd27 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:57.057421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe7f38-e783-4bdb-9216-58b708e0dfd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:57.058409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe7f38-e783-4bdb-9216-58b708e0dfd4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:57.058769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe7f38-e783-4bdb-9216-58b708e0dfd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:57.059157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96fe7f38-e783-4bdb-9216-58b708e0dfd4 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:57.060195 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe7f38-e783-4bdb-9216-58b708e0dfd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:57.060643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe7f38-e783-4bdb-9216-58b708e0dfd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:57.062118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96fe7f38-e783-4bdb-9216-58b708e0dfd4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:57.078709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 633/1257] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:56:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:57.084549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f540998-5ad0-4da0-8a08-b1e364d55003 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:57.225357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f540998-5ad0-4da0-8a08-b1e364d55003 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:57.282969 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40cf64-d948-441b-8529-94005d167325 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:57.283860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40cf64-d948-441b-8529-94005d167325 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:57.284257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40cf64-d948-441b-8529-94005d167325 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:57.284610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f40cf64-d948-441b-8529-94005d167325 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:57.292754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f40cf64-d948-441b-8529-94005d167325 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:57.294253 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f40cf64-d948-441b-8529-94005d167325 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:57.311406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-5f40cf64-d948-441b-8529-94005d167325 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:56:57.311406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:56:57.311406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:56:57.311406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:56:57.311406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:56:57.311406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:56:57.311406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:56:57.311406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:56:57.311406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:56:57.311406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:56:57.311406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:56:57.311406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:56:57.311406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:56:57.311406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:56:57.311406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:56:57.312630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:56:57.312630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:56:57.312630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:56:57.312630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:56:57.312630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:56:57.312630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:56:57.312630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:56:57.312630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:56:57.312630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:56:57.312630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:56:57.312630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:56:57.312630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:56:57.312630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:56:57.312630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:56:57.312630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:56:57.312630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:56:57.312630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:56:57.319689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:56:57.319689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:56:57.319689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:56:57.319689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:56:57.319689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:56:57.319689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:56:57.319689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:56:57.319689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:56:57.319689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 625/1258] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:56:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:56:57.320606 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe7f38-e783-4bdb-9216-58b708e0dfd4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:57.451371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96fe7f38-e783-4bdb-9216-58b708e0dfd4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:57.492970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f363c5b-ce54-472c-b089-f6a617478903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:57.493597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f363c5b-ce54-472c-b089-f6a617478903 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:57.493931 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f363c5b-ce54-472c-b089-f6a617478903 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:57.494175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f363c5b-ce54-472c-b089-f6a617478903 None admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:57.495520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f363c5b-ce54-472c-b089-f6a617478903 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:57.496250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f363c5b-ce54-472c-b089-f6a617478903 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:57.497759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f363c5b-ce54-472c-b089-f6a617478903 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:57.514156 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 634/1259] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:56:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:57.520928 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40cf64-d948-441b-8529-94005d167325 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:57.552753 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:56:57.641387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f40cf64-d948-441b-8529-94005d167325 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:56:57.706604 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c363b3f-e99c-4881-9125-07f0daed2dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:56:57.707546 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c363b3f-e99c-4881-9125-07f0daed2dcf None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:57.707959 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c363b3f-e99c-4881-9125-07f0daed2dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:57.708434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c363b3f-e99c-4881-9125-07f0daed2dcf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:57.710072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c363b3f-e99c-4881-9125-07f0daed2dcf None admin] RBAC: Adding query filter params (role.id=6964639a7f6c41e8a6701ef533151c41, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=ee29b7f1cea64d6781e1df5c8021363f) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:56:57.710909 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c363b3f-e99c-4881-9125-07f0daed2dcf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:56:57.713692 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c363b3f-e99c-4881-9125-07f0daed2dcf None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:56:57.724928 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 626/1260] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:56:57 2022] GET /identity/v3/role_assignments?user.id=ee29b7f1cea64d6781e1df5c8021363f&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 259 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:56:59.812167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fe78b-70d0-446c-8f4c-2e203a6d74ce None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:59.812780 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fe78b-70d0-446c-8f4c-2e203a6d74ce None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:59.812780 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-465fe78b-70d0-446c-8f4c-2e203a6d74ce None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:59.814192 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 635/1261] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:56:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:56:59.822518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e812c78-ee0d-4020-90cd-0389fee52560 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:56:59.822518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e812c78-ee0d-4020-90cd-0389fee52560 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:56:59.822518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e812c78-ee0d-4020-90cd-0389fee52560 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:56:59.840100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8e812c78-ee0d-4020-90cd-0389fee52560 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:56:59.841310 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8e812c78-ee0d-4020-90cd-0389fee52560 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:56:59.912613 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 627/1262] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:56:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:56:59.935932 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465fe78b-70d0-446c-8f4c-2e203a6d74ce None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:56:59.947292 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:00.040633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-465fe78b-70d0-446c-8f4c-2e203a6d74ce None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:00.079576 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7850de-bb94-4870-8bfe-2ac0c012bbfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:00.080376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7850de-bb94-4870-8bfe-2ac0c012bbfc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:00.080654 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7850de-bb94-4870-8bfe-2ac0c012bbfc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:00.080928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c7850de-bb94-4870-8bfe-2ac0c012bbfc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:00.091617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c7850de-bb94-4870-8bfe-2ac0c012bbfc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:57:00.099725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-9c7850de-bb94-4870-8bfe-2ac0c012bbfc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:57:00.099725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:57:00.099725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:57:00.099725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:57:00.099725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:57:00.099725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:57:00.099725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:57:00.099725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:57:00.099725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:57:00.099725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:57:00.099725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:57:00.099725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:57:00.099725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:57:00.099725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:57:00.099725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:57:00.099725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:57:00.100893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:57:00.100893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:57:00.100893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:57:00.100893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:57:00.100893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:57:00.100893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:57:00.100893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:57:00.100893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:57:00.100893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:57:00.100893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:57:00.100893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:57:00.100893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:57:00.100893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:57:00.100893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:57:00.100893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:57:00.100893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:57:00.101927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:57:00.101927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:57:00.101927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:57:00.101927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:57:00.101927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:57:00.101927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:57:00.101927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:57:00.101927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:57:00.101927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:57:00.101927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:57:00.101927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:57:00.101927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:57:00.101927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 636/1263] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:56:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:57:00.108569 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e812c78-ee0d-4020-90cd-0389fee52560 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:00.127742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:00.211576 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e812c78-ee0d-4020-90cd-0389fee52560 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:00.248010 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61deafa1-326d-4234-bd92-21e148b82166 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:00.248681 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61deafa1-326d-4234-bd92-21e148b82166 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:00.248971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61deafa1-326d-4234-bd92-21e148b82166 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:00.249252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61deafa1-326d-4234-bd92-21e148b82166 None admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:00.250060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61deafa1-326d-4234-bd92-21e148b82166 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:57:00.250398 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61deafa1-326d-4234-bd92-21e148b82166 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:00.251931 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61deafa1-326d-4234-bd92-21e148b82166 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:00.261752 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 628/1264] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:57:00 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:57:00.267819 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7850de-bb94-4870-8bfe-2ac0c012bbfc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:00.347173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:00.382239 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c7850de-bb94-4870-8bfe-2ac0c012bbfc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:00.417177 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef8261-bf9e-40b6-8539-30ce1497f956 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:00.417980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef8261-bf9e-40b6-8539-30ce1497f956 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:00.418267 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef8261-bf9e-40b6-8539-30ce1497f956 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:00.418581 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cef8261-bf9e-40b6-8539-30ce1497f956 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:00.429989 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef8261-bf9e-40b6-8539-30ce1497f956 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:00.431491 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cef8261-bf9e-40b6-8539-30ce1497f956 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:00.449867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-0cef8261-bf9e-40b6-8539-30ce1497f956 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:57:00.449867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:57:00.449867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:57:00.449867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:57:00.449867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:57:00.449867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:57:00.449867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:57:00.449867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:57:00.449867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:57:00.449867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:57:00.449867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:57:00.449867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:57:00.449867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:57:00.449867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:57:00.449867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:57:00.451049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:57:00.451049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:57:00.451049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:57:00.451049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:57:00.451049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:57:00.451049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:57:00.451049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:57:00.451049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:57:00.451049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:57:00.451049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:57:00.451049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:57:00.451049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:57:00.451049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:57:00.451049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:57:00.451049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:57:00.451049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:57:00.451049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:57:00.452282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:57:00.452282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:57:00.452282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:57:00.452282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:57:00.452282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 637/1265] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:57:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:57:00.457008 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61deafa1-326d-4234-bd92-21e148b82166 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:00.560413 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:00.593813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61deafa1-326d-4234-bd92-21e148b82166 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:00.629138 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff37ca-ea5b-4c5a-9e5a-e54ad0235ee1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:00.629735 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff37ca-ea5b-4c5a-9e5a-e54ad0235ee1 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:00.630024 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff37ca-ea5b-4c5a-9e5a-e54ad0235ee1 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:00.630329 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff37ca-ea5b-4c5a-9e5a-e54ad0235ee1 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:00.631072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff37ca-ea5b-4c5a-9e5a-e54ad0235ee1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:57:00.631385 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff37ca-ea5b-4c5a-9e5a-e54ad0235ee1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:00.632764 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcff37ca-ea5b-4c5a-9e5a-e54ad0235ee1 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:00.645678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 629/1266] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:57:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:57:00.651547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef8261-bf9e-40b6-8539-30ce1497f956 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:00.756145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cef8261-bf9e-40b6-8539-30ce1497f956 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:00.781051 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:00.809647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b171f0-e274-4b46-bb78-0a656458c361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:00.810668 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b171f0-e274-4b46-bb78-0a656458c361 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:00.811052 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b171f0-e274-4b46-bb78-0a656458c361 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:00.811427 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b171f0-e274-4b46-bb78-0a656458c361 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:00.826812 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b171f0-e274-4b46-bb78-0a656458c361 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:00.829869 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b171f0-e274-4b46-bb78-0a656458c361 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:00.866570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-86b171f0-e274-4b46-bb78-0a656458c361 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 13:57:00.866570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:57:00.866570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:57:00.866570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:57:00.866570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:57:00.866570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:57:00.866570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:57:00.866570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:57:00.866570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:57:00.866570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:57:00.866570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:57:00.866570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:57:00.866570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:57:00.866570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:57:00.866570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:57:00.866570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:57:00.867617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:57:00.867617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:57:00.867617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:57:00.867617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:57:00.867617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:57:00.867617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:57:00.867617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:57:00.867617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:57:00.867617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:57:00.867617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:57:00.867617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:57:00.867617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:57:00.867617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:57:00.867617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:57:00.867617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:57:00.867617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:57:00.867617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:57:00.867617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:57:00.868574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:57:00.868574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:57:00.868574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:57:00.868574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:57:00.868574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:57:00.868574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:57:00.868574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:57:00.868574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:57:00.868574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 13:57:00.868574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:57:00.869119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 638/1267] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:57:00 2022] GET /identity/v3/users/placement => generated 87 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:57:00.875864 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff37ca-ea5b-4c5a-9e5a-e54ad0235ee1 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:00.981337 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff37ca-ea5b-4c5a-9e5a-e54ad0235ee1 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:01.017990 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb97bc-ced6-43bd-9ff8-5b219cba0d9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:01.018690 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb97bc-ced6-43bd-9ff8-5b219cba0d9d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:01.018975 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb97bc-ced6-43bd-9ff8-5b219cba0d9d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:01.019252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befb97bc-ced6-43bd-9ff8-5b219cba0d9d None admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:01.020105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb97bc-ced6-43bd-9ff8-5b219cba0d9d None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:57:01.020419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb97bc-ced6-43bd-9ff8-5b219cba0d9d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:01.021868 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-befb97bc-ced6-43bd-9ff8-5b219cba0d9d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:01.031180 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:01.040528 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 630/1268] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Jul 11 13:57:00 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:57:01.046784 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b171f0-e274-4b46-bb78-0a656458c361 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:01.150189 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b171f0-e274-4b46-bb78-0a656458c361 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:01.185951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8e585-18bd-465f-af2f-5b91de1724c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:01.186767 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8e585-18bd-465f-af2f-5b91de1724c7 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:01.187053 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8e585-18bd-465f-af2f-5b91de1724c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:01.187328 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a8e585-18bd-465f-af2f-5b91de1724c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:01.198179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8e585-18bd-465f-af2f-5b91de1724c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:01.199670 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a8e585-18bd-465f-af2f-5b91de1724c7 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:01.217321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-15a8e585-18bd-465f-af2f-5b91de1724c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:57:01.217321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:57:01.217321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:57:01.217321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:57:01.217321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:57:01.217321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:57:01.217321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:57:01.217321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:57:01.217321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:57:01.217321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:57:01.217321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:57:01.217321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:57:01.217321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:57:01.217321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:57:01.217321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:57:01.218597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:57:01.218597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:57:01.218597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:57:01.218597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:57:01.218597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:57:01.218597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:57:01.218597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:57:01.218597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:57:01.218597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:57:01.218597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:57:01.218597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:57:01.218597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:57:01.218597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:57:01.218597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:57:01.218597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:57:01.218597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:57:01.218597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:57:01.219488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:57:01.219488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:57:01.219488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:57:01.219488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:57:01.219488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 639/1269] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:57:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:57:01.225755 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb97bc-ced6-43bd-9ff8-5b219cba0d9d None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:01.349758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befb97bc-ced6-43bd-9ff8-5b219cba0d9d None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:01.385549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a61853-ecdb-4784-95e6-39a25c284fe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:01.386222 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a61853-ecdb-4784-95e6-39a25c284fe2 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:01.386542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a61853-ecdb-4784-95e6-39a25c284fe2 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:01.386825 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a61853-ecdb-4784-95e6-39a25c284fe2 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:01.387554 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a61853-ecdb-4784-95e6-39a25c284fe2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:57:01.387863 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a61853-ecdb-4784-95e6-39a25c284fe2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:01.389249 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a61853-ecdb-4784-95e6-39a25c284fe2 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:01.402362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 631/1270] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:57:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:57:01.408240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8e585-18bd-465f-af2f-5b91de1724c7 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:01.444486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:01.520015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a8e585-18bd-465f-af2f-5b91de1724c7 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:01.557211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4922e-055a-458f-af1d-be8f111b3e6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:01.558014 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4922e-055a-458f-af1d-be8f111b3e6a None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:01.558323 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4922e-055a-458f-af1d-be8f111b3e6a None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:01.558608 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fa4922e-055a-458f-af1d-be8f111b3e6a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:01.565724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4922e-055a-458f-af1d-be8f111b3e6a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:01.567210 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fa4922e-055a-458f-af1d-be8f111b3e6a None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:01.582417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-5fa4922e-055a-458f-af1d-be8f111b3e6a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:57:01.582417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:57:01.582417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:57:01.582417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:57:01.582417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:57:01.582417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:57:01.582417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:57:01.582417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:57:01.582417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:57:01.582417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:57:01.582417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:57:01.582417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:57:01.582417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:57:01.582417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:57:01.582417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:57:01.583613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:57:01.583613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:57:01.583613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:57:01.583613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:57:01.583613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:57:01.583613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:57:01.583613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:57:01.583613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:57:01.583613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:57:01.583613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:57:01.583613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:57:01.583613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:57:01.583613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:57:01.583613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:57:01.583613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:57:01.583613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:57:01.583613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:57:01.590150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:57:01.590150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:57:01.590150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:57:01.590150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:57:01.590150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:57:01.590150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:57:01.590150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:57:01.590150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:57:01.590150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 640/1271] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:57:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:57:01.590758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a61853-ecdb-4784-95e6-39a25c284fe2 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:01.672874 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:01.720991 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a61853-ecdb-4784-95e6-39a25c284fe2 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:01.758048 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b937324d-72f6-4434-871a-ca1e6414cfee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:01.758754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937324d-72f6-4434-871a-ca1e6414cfee None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:01.759042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937324d-72f6-4434-871a-ca1e6414cfee None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:01.759322 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b937324d-72f6-4434-871a-ca1e6414cfee None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:01.760133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937324d-72f6-4434-871a-ca1e6414cfee None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:57:01.760507 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937324d-72f6-4434-871a-ca1e6414cfee None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:01.762242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b937324d-72f6-4434-871a-ca1e6414cfee None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:01.783665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 632/1272] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:57:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:57:01.787121 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4922e-055a-458f-af1d-be8f111b3e6a None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:01.894476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:01.903522 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fa4922e-055a-458f-af1d-be8f111b3e6a None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:01.939421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac232825-4529-47ae-974b-db744bd82ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:01.940234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac232825-4529-47ae-974b-db744bd82ee9 None admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:01.940521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac232825-4529-47ae-974b-db744bd82ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:01.940801 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac232825-4529-47ae-974b-db744bd82ee9 None admin] PATH_INFO: `/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/ee29b7f1cea64d6781e1df5c8021363f/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:01.942797 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac232825-4529-47ae-974b-db744bd82ee9 None admin] RBAC: Authorizing `identity:create_grant(project_id=08ee65f8b338491f80e7beac5e316b23, user_id=ee29b7f1cea64d6781e1df5c8021363f, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:01.944218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac232825-4529-47ae-974b-db744bd82ee9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:01.957820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 641/1273] 10.222.0.49 () {58 vars in 1439 bytes} [Mon Jul 11 13:57:01 2022] PUT /identity/v3/projects/08ee65f8b338491f80e7beac5e316b23/users/ee29b7f1cea64d6781e1df5c8021363f/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 13:57:04.038120 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa82124-6611-4210-8aa6-41edf1cd9cc7 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:04.038750 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa82124-6611-4210-8aa6-41edf1cd9cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:04.038750 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3aa82124-6611-4210-8aa6-41edf1cd9cc7 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:04.040125 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 633/1274] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:57:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:57:04.046092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c191e-4ac7-4bbd-af7b-1638fa110562 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:04.046823 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c191e-4ac7-4bbd-af7b-1638fa110562 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:04.046823 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314c191e-4ac7-4bbd-af7b-1638fa110562 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:04.065188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-314c191e-4ac7-4bbd-af7b-1638fa110562 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:57:04.066437 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-314c191e-4ac7-4bbd-af7b-1638fa110562 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:57:04.140151 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 642/1275] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:57:04 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:57:04.164176 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa82124-6611-4210-8aa6-41edf1cd9cc7 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:04.273213 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3aa82124-6611-4210-8aa6-41edf1cd9cc7 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:04.293166 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:04.317874 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5eda33-c50d-478e-b6d9-4efcc10b5635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:04.318719 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5eda33-c50d-478e-b6d9-4efcc10b5635 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:04.319014 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5eda33-c50d-478e-b6d9-4efcc10b5635 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:04.319293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5eda33-c50d-478e-b6d9-4efcc10b5635 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:04.332929 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5eda33-c50d-478e-b6d9-4efcc10b5635 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 11 13:57:04.341397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-4a5eda33-c50d-478e-b6d9-4efcc10b5635 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:57:04.341397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:57:04.341397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:57:04.341397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:57:04.341397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:57:04.341397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:57:04.341397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:57:04.341397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:57:04.341397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:57:04.341397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:57:04.341397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:57:04.341397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:57:04.341397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 11 13:57:04.341397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 11 13:57:04.341397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 11 13:57:04.341397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 11 13:57:04.342646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 11 13:57:04.342646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 11 13:57:04.342646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 11 13:57:04.342646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 11 13:57:04.342646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:57:04.342646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:57:04.342646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:57:04.342646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:57:04.342646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:57:04.342646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:57:04.342646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:57:04.342646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:57:04.342646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:57:04.342646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:57:04.342646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:57:04.342646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:57:04.343556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:57:04.343556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:57:04.343556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:57:04.343556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:57:04.343556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 11 13:57:04.343556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 11 13:57:04.343556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 11 13:57:04.343556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 11 13:57:04.343556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 11 13:57:04.343556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 11 13:57:04.343556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 11 13:57:04.343556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:57:04.343556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 634/1276] 10.222.0.49 () {56 vars in 1096 bytes} [Mon Jul 11 13:57:04 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:57:04.351317 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c191e-4ac7-4bbd-af7b-1638fa110562 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:04.439684 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:04.462372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314c191e-4ac7-4bbd-af7b-1638fa110562 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:04.498803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f10882-8d62-4a69-a159-dff2dc59d7bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:04.499478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f10882-8d62-4a69-a159-dff2dc59d7bc None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:04.499761 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f10882-8d62-4a69-a159-dff2dc59d7bc None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:04.500044 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f10882-8d62-4a69-a159-dff2dc59d7bc None admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:04.500805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f10882-8d62-4a69-a159-dff2dc59d7bc None admin] RBAC: Adding query filter params (name=admin) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:57:04.501113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f10882-8d62-4a69-a159-dff2dc59d7bc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:04.502538 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f10882-8d62-4a69-a159-dff2dc59d7bc None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:04.512252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 643/1277] 10.222.0.49 () {56 vars in 1099 bytes} [Mon Jul 11 13:57:04 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:57:04.518609 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5eda33-c50d-478e-b6d9-4efcc10b5635 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:04.627985 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5eda33-c50d-478e-b6d9-4efcc10b5635 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:04.663709 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaae155-b9c4-4e5d-88db-4720f9b6d3fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:04.664518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaae155-b9c4-4e5d-88db-4720f9b6d3fa None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:04.664800 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaae155-b9c4-4e5d-88db-4720f9b6d3fa None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:04.665079 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaae155-b9c4-4e5d-88db-4720f9b6d3fa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:04.676923 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaae155-b9c4-4e5d-88db-4720f9b6d3fa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:04.678462 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaae155-b9c4-4e5d-88db-4720f9b6d3fa None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:04.697521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-feaae155-b9c4-4e5d-88db-4720f9b6d3fa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:57:04.697521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:57:04.697521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:57:04.697521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:57:04.697521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:57:04.697521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:57:04.697521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:57:04.697521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:57:04.697521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:57:04.697521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:57:04.697521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:57:04.697521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:57:04.697521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:57:04.697521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:57:04.697521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:57:04.700097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:57:04.700097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:57:04.700097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:57:04.700097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:57:04.700097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:57:04.700097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:57:04.700097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:57:04.700097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:57:04.700097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:57:04.700097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:57:04.700097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:57:04.700097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:57:04.700097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:57:04.700097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:57:04.700097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:57:04.700097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:57:04.700097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:57:04.701763 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:57:04.701763 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:57:04.701763 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:57:04.701763 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:57:04.701763 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 635/1278] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:57:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:57:04.705571 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f10882-8d62-4a69-a159-dff2dc59d7bc None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:04.873576 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f10882-8d62-4a69-a159-dff2dc59d7bc None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:04.909516 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731bfb1-edb4-412b-8d8d-62c8fa9a87d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:04.910201 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731bfb1-edb4-412b-8d8d-62c8fa9a87d9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:04.910520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731bfb1-edb4-412b-8d8d-62c8fa9a87d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:04.910802 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7731bfb1-edb4-412b-8d8d-62c8fa9a87d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:04.911550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731bfb1-edb4-412b-8d8d-62c8fa9a87d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:57:04.911862 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731bfb1-edb4-412b-8d8d-62c8fa9a87d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:04.913263 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7731bfb1-edb4-412b-8d8d-62c8fa9a87d9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:04.927361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 644/1279] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:57:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:57:04.935452 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaae155-b9c4-4e5d-88db-4720f9b6d3fa None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:04.971075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:05.051106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaae155-b9c4-4e5d-88db-4720f9b6d3fa None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:05.088558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ba07a-0ae8-47c2-88a3-ac9d80aa2bda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:05.089372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ba07a-0ae8-47c2-88a3-ac9d80aa2bda None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:05.089655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ba07a-0ae8-47c2-88a3-ac9d80aa2bda None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:05.089930 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90ba07a-0ae8-47c2-88a3-ac9d80aa2bda None admin] PATH_INFO: `/v3/users/placement` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:05.099249 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ba07a-0ae8-47c2-88a3-ac9d80aa2bda None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:05.100708 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90ba07a-0ae8-47c2-88a3-ac9d80aa2bda None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:05.116865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-b90ba07a-0ae8-47c2-88a3-ac9d80aa2bda None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 11 13:57:05.116865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:57:05.116865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:57:05.116865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:57:05.116865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:57:05.116865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:57:05.116865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:57:05.116865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:57:05.116865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:57:05.116865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:57:05.116865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:57:05.116865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:57:05.116865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 11 13:57:05.116865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 11 13:57:05.116865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 11 13:57:05.116865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 11 13:57:05.118175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:57:05.118175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:57:05.118175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 11 13:57:05.118175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:57:05.118175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 11 13:57:05.118175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 11 13:57:05.118175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:57:05.118175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:57:05.118175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:57:05.118175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:57:05.118175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:57:05.118175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:57:05.118175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:57:05.118175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:57:05.118175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:57:05.118175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:57:05.118175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:57:05.118175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:57:05.119238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:57:05.119238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:57:05.119238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 11 13:57:05.119238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 11 13:57:05.119238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 11 13:57:05.119238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 11 13:57:05.119238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 11 13:57:05.119238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 11 13:57:05.119238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 11 13:57:05.119238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:57:05.119238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 636/1280] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:57:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 11 13:57:05.130143 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731bfb1-edb4-412b-8d8d-62c8fa9a87d9 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:05.134372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:05.240910 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7731bfb1-edb4-412b-8d8d-62c8fa9a87d9 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:05.284314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32289bef-f77e-4716-8640-3de96f2b626c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:05.284997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32289bef-f77e-4716-8640-3de96f2b626c None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:05.285284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32289bef-f77e-4716-8640-3de96f2b626c None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:05.285566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32289bef-f77e-4716-8640-3de96f2b626c None admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:05.286459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32289bef-f77e-4716-8640-3de96f2b626c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:57:05.286775 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32289bef-f77e-4716-8640-3de96f2b626c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:05.288215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32289bef-f77e-4716-8640-3de96f2b626c None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:05.308895 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 645/1281] 10.222.0.49 () {56 vars in 1143 bytes} [Mon Jul 11 13:57:05 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:57:05.317856 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ba07a-0ae8-47c2-88a3-ac9d80aa2bda None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:05.417185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:05.428753 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90ba07a-0ae8-47c2-88a3-ac9d80aa2bda None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:05.465291 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354e3fa-6e2e-4c46-a0a4-b643fc421134 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:05.465956 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354e3fa-6e2e-4c46-a0a4-b643fc421134 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:05.466239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354e3fa-6e2e-4c46-a0a4-b643fc421134 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:05.466548 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e354e3fa-6e2e-4c46-a0a4-b643fc421134 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:05.541867 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354e3fa-6e2e-4c46-a0a4-b643fc421134 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:05.544815 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e354e3fa-6e2e-4c46-a0a4-b643fc421134 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:05.638749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:05.650256 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 637/1282] 10.222.0.49 () {58 vars in 1282 bytes} [Mon Jul 11 13:57:05 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 11 13:57:05.657009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32289bef-f77e-4716-8640-3de96f2b626c None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:05.721024 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:05.761464 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32289bef-f77e-4716-8640-3de96f2b626c None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:05.802004 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270029e3-9123-474b-a32b-2d256b975fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:05.802757 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270029e3-9123-474b-a32b-2d256b975fd4 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:05.803048 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270029e3-9123-474b-a32b-2d256b975fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:05.803330 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-270029e3-9123-474b-a32b-2d256b975fd4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:05.814427 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270029e3-9123-474b-a32b-2d256b975fd4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:05.815882 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-270029e3-9123-474b-a32b-2d256b975fd4 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:05.835596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-270029e3-9123-474b-a32b-2d256b975fd4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:57:05.835596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:57:05.835596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:57:05.835596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:57:05.835596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:57:05.835596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:57:05.835596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:57:05.835596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:57:05.835596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:57:05.835596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:57:05.835596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:57:05.835596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:57:05.835596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 11 13:57:05.835596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 11 13:57:05.835596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 11 13:57:05.836803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 11 13:57:05.836803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:57:05.836803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:57:05.836803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:57:05.836803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:57:05.836803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:57:05.836803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:57:05.836803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:57:05.836803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:57:05.836803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:57:05.836803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:57:05.836803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:57:05.836803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:57:05.836803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:57:05.836803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:57:05.836803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:57:05.836803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:57:05.837924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 11 13:57:05.837924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 11 13:57:05.837924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 11 13:57:05.837924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:57:05.837924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 646/1283] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 13:57:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:57:05.843269 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354e3fa-6e2e-4c46-a0a4-b643fc421134 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:05.947478 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e354e3fa-6e2e-4c46-a0a4-b643fc421134 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:05.983921 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5ae87-020d-47f4-b914-67cc4e6b5951 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:05.984589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5ae87-020d-47f4-b914-67cc4e6b5951 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:05.984883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5ae87-020d-47f4-b914-67cc4e6b5951 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:05.985171 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5ae87-020d-47f4-b914-67cc4e6b5951 None admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:05.985919 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5ae87-020d-47f4-b914-67cc4e6b5951 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:57:05.986259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5ae87-020d-47f4-b914-67cc4e6b5951 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:05.987704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5ae87-020d-47f4-b914-67cc4e6b5951 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:06.000884 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 638/1284] 10.222.0.49 () {56 vars in 1109 bytes} [Mon Jul 11 13:57:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:57:06.008675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270029e3-9123-474b-a32b-2d256b975fd4 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:06.113960 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-270029e3-9123-474b-a32b-2d256b975fd4 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:06.130071 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:06.151739 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b907031-caee-454f-8cd5-c77c16abaf48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:06.152537 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b907031-caee-454f-8cd5-c77c16abaf48 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:06.152816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b907031-caee-454f-8cd5-c77c16abaf48 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:06.153090 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b907031-caee-454f-8cd5-c77c16abaf48 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:06.160195 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b907031-caee-454f-8cd5-c77c16abaf48 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:06.161655 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b907031-caee-454f-8cd5-c77c16abaf48 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:06.176556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application [None req-2b907031-caee-454f-8cd5-c77c16abaf48 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:57:06.176556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:57:06.176556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:57:06.176556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:57:06.176556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:57:06.176556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:57:06.176556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:57:06.176556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:57:06.176556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:57:06.176556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:57:06.176556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:57:06.176556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:57:06.176556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 11 13:57:06.176556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 11 13:57:06.176556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 11 13:57:06.177572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 11 13:57:06.177572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:57:06.177572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:57:06.177572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:57:06.177572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:57:06.177572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:57:06.177572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:57:06.177572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:57:06.177572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application with Lock( Jul 11 13:57:06.177572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:57:06.177572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:57:06.177572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:57:06.177572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:57:06.177572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:57:06.177572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:57:06.177572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:57:06.177572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:57:06.178627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 11 13:57:06.178627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 11 13:57:06.178627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 11 13:57:06.178627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 11 13:57:06.178627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 11 13:57:06.178627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 11 13:57:06.178627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 11 13:57:06.178627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: ERROR keystone.server.flask.application Jul 11 13:57:06.178627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 647/1285] 10.222.0.49 () {56 vars in 1111 bytes} [Mon Jul 11 13:57:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:57:06.184683 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5ae87-020d-47f4-b914-67cc4e6b5951 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:06.244393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:06.297417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5ae87-020d-47f4-b914-67cc4e6b5951 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:06.335347 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e429d2-fa7f-46d3-a177-9675b48a52d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:06.336077 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e429d2-fa7f-46d3-a177-9675b48a52d8 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:06.336412 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e429d2-fa7f-46d3-a177-9675b48a52d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:06.336739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e429d2-fa7f-46d3-a177-9675b48a52d8 None admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:06.337615 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e429d2-fa7f-46d3-a177-9675b48a52d8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:57:06.337979 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e429d2-fa7f-46d3-a177-9675b48a52d8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:06.340559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4e429d2-fa7f-46d3-a177-9675b48a52d8 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:06.354600 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 639/1286] 10.222.0.49 () {56 vars in 1148 bytes} [Mon Jul 11 13:57:06 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:57:06.360925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b907031-caee-454f-8cd5-c77c16abaf48 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:06.469351 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b907031-caee-454f-8cd5-c77c16abaf48 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:06.504797 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343e41de-d622-489c-8bd9-90dff4531b3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:06.505470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e41de-d622-489c-8bd9-90dff4531b3b None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:06.505758 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e41de-d622-489c-8bd9-90dff4531b3b None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:06.506037 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343e41de-d622-489c-8bd9-90dff4531b3b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:06.506924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e41de-d622-489c-8bd9-90dff4531b3b None admin] RBAC: Adding query filter params (role.id=6964639a7f6c41e8a6701ef533151c41, scope.project.id=08ee65f8b338491f80e7beac5e316b23, user.id=ee29b7f1cea64d6781e1df5c8021363f) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:57:06.507238 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e41de-d622-489c-8bd9-90dff4531b3b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:06.508680 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343e41de-d622-489c-8bd9-90dff4531b3b None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:06.515611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 648/1287] 10.222.0.49 () {56 vars in 1374 bytes} [Mon Jul 11 13:57:06 2022] GET /identity/v3/role_assignments?user.id=ee29b7f1cea64d6781e1df5c8021363f&scope.project.id=08ee65f8b338491f80e7beac5e316b23&role.id=6964639a7f6c41e8a6701ef533151c41 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:57:08.623748 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2653f7-21b7-444b-9593-39f29e07ee9e None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:08.624292 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2653f7-21b7-444b-9593-39f29e07ee9e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:08.624292 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba2653f7-21b7-444b-9593-39f29e07ee9e None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:08.625688 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 640/1288] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:57:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:57:08.631960 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3565cb-4556-4b8e-9c1d-198351a6c770 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:08.632441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3565cb-4556-4b8e-9c1d-198351a6c770 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:08.632546 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3565cb-4556-4b8e-9c1d-198351a6c770 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:08.653241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1f3565cb-4556-4b8e-9c1d-198351a6c770 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:57:08.654490 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1f3565cb-4556-4b8e-9c1d-198351a6c770 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:57:08.696973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:08.741855 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 649/1289] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:57:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 11 13:57:08.765756 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2653f7-21b7-444b-9593-39f29e07ee9e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:08.899270 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba2653f7-21b7-444b-9593-39f29e07ee9e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:08.911199 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:08.936607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c9aeb-ae23-4dcc-bc22-ae5078ab7965 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:08.937412 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c9aeb-ae23-4dcc-bc22-ae5078ab7965 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:08.937699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c9aeb-ae23-4dcc-bc22-ae5078ab7965 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:08.937978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-363c9aeb-ae23-4dcc-bc22-ae5078ab7965 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:08.938679 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c9aeb-ae23-4dcc-bc22-ae5078ab7965 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:08.940089 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-363c9aeb-ae23-4dcc-bc22-ae5078ab7965 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:08.952807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-363c9aeb-ae23-4dcc-bc22-ae5078ab7965 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 13:57:08.952807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:57:08.952807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:57:08.952807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:57:08.952807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:57:08.952807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:57:08.952807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:57:08.952807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:57:08.952807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 13:57:08.952807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:57:08.952807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:57:08.952807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:57:08.952807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:57:08.952807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:57:08.952807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:57:08.952807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:57:08.952807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:57:08.954082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:57:08.954082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:57:08.954082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:57:08.954082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:57:08.954082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:57:08.954082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:57:08.954082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:57:08.954082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:57:08.954082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:57:08.954082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:57:08.954082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:57:08.954082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:57:08.954082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:57:08.954082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:57:08.954082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:57:08.954082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:57:08.955382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:57:08.955382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:57:08.955382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:57:08.955382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:57:08.955382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:57:08.955382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:57:08.955382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:57:08.955382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:57:08.955382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:57:08.955382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:57:08.955382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:57:08.955382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:57:08.955382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 13:57:08.955382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:57:08.955382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 641/1290] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 11 13:57:08 2022] GET /identity/v3/services/placement => generated 90 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:57:08.960070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3565cb-4556-4b8e-9c1d-198351a6c770 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:09.059175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3565cb-4556-4b8e-9c1d-198351a6c770 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:09.098190 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd72aa60-ca2a-4731-9c96-071829b89849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:09.098886 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72aa60-ca2a-4731-9c96-071829b89849 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:09.099173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72aa60-ca2a-4731-9c96-071829b89849 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:09.099452 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd72aa60-ca2a-4731-9c96-071829b89849 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:09.100175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72aa60-ca2a-4731-9c96-071829b89849 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:57:09.100489 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72aa60-ca2a-4731-9c96-071829b89849 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:09.101867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd72aa60-ca2a-4731-9c96-071829b89849 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:09.107290 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 650/1291] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Jul 11 13:57:08 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:57:09.115161 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c9aeb-ae23-4dcc-bc22-ae5078ab7965 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:09.225542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-363c9aeb-ae23-4dcc-bc22-ae5078ab7965 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:09.264401 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57332e52-e68d-40d3-955c-f046b732c99d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:09.265086 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57332e52-e68d-40d3-955c-f046b732c99d None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:09.265374 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57332e52-e68d-40d3-955c-f046b732c99d None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:09.265662 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57332e52-e68d-40d3-955c-f046b732c99d None admin] PATH_INFO: `/v3/services` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:09.266428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57332e52-e68d-40d3-955c-f046b732c99d None admin] RBAC: Adding query filter params (type=placement) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:57:09.266743 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57332e52-e68d-40d3-955c-f046b732c99d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:09.268149 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57332e52-e68d-40d3-955c-f046b732c99d None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:09.273572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 642/1292] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Jul 11 13:57:09 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:57:11.312667 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99536787-9add-4c08-af3e-962b86e32d8f None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:11.313187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99536787-9add-4c08-af3e-962b86e32d8f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:11.313301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99536787-9add-4c08-af3e-962b86e32d8f None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:11.314731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 651/1293] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:57:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:57:11.319841 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea3e76-e077-427c-adf4-8a9f44a049a8 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:11.320249 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea3e76-e077-427c-adf4-8a9f44a049a8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:11.320393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3ea3e76-e077-427c-adf4-8a9f44a049a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:11.336779 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a3ea3e76-e077-427c-adf4-8a9f44a049a8 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:57:11.337886 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a3ea3e76-e077-427c-adf4-8a9f44a049a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:57:11.368555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:11.404684 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 643/1294] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:57:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:57:11.427643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99536787-9add-4c08-af3e-962b86e32d8f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:11.459755 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:11.534824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99536787-9add-4c08-af3e-962b86e32d8f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:11.570230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd7684e-015f-4872-b5b9-6c0ca4543da9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:11.570939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7684e-015f-4872-b5b9-6c0ca4543da9 None admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:11.571231 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7684e-015f-4872-b5b9-6c0ca4543da9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:11.571515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd7684e-015f-4872-b5b9-6c0ca4543da9 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:11.572271 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7684e-015f-4872-b5b9-6c0ca4543da9 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:11.574152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd7684e-015f-4872-b5b9-6c0ca4543da9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:11.589751 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 652/1295] 10.222.0.49 () {60 vars in 1141 bytes} [Mon Jul 11 13:57:11 2022] POST /identity/v3/services => generated 246 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 13:57:13.654470 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f440e-2390-4f87-8a24-e780fa0c19ee None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:13.655028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f440e-2390-4f87-8a24-e780fa0c19ee None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:13.655028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659f440e-2390-4f87-8a24-e780fa0c19ee None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:13.656404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 644/1296] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:57:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:57:13.661535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96a59b-2769-46e6-863f-dca99696e5b3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:13.661906 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96a59b-2769-46e6-863f-dca99696e5b3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:13.662088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab96a59b-2769-46e6-863f-dca99696e5b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:13.679089 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ab96a59b-2769-46e6-863f-dca99696e5b3 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:57:13.680546 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ab96a59b-2769-46e6-863f-dca99696e5b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:57:13.750796 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 653/1297] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:57:13 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:57:13.776649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f440e-2390-4f87-8a24-e780fa0c19ee None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:13.894493 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659f440e-2390-4f87-8a24-e780fa0c19ee None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:13.929652 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c6316-e51e-4fb1-85e5-180555df23ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:13.930397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c6316-e51e-4fb1-85e5-180555df23ef None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:13.930682 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c6316-e51e-4fb1-85e5-180555df23ef None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:13.930959 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e09c6316-e51e-4fb1-85e5-180555df23ef None admin] PATH_INFO: `/v3/services/placement` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:13.931623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c6316-e51e-4fb1-85e5-180555df23ef None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:13.934093 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e09c6316-e51e-4fb1-85e5-180555df23ef None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:13.946075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-e09c6316-e51e-4fb1-85e5-180555df23ef None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 13:57:13.946075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:57:13.946075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:57:13.946075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:57:13.946075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:57:13.946075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:57:13.946075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:57:13.946075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:57:13.946075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 13:57:13.946075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:57:13.946075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:57:13.946075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:57:13.946075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:57:13.946075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:57:13.946075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:57:13.946075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:57:13.946075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:57:13.947911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:57:13.947911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:57:13.947911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:57:13.947911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:57:13.947911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:57:13.947911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:57:13.947911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:57:13.947911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:57:13.947911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:57:13.947911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:57:13.947911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:57:13.947911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:57:13.947911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:57:13.947911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:57:13.947911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:57:13.947911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:57:13.948971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:57:13.948971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:57:13.948971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:57:13.948971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:57:13.948971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:57:13.948971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:57:13.948971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:57:13.948971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:57:13.948971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:57:13.948971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:57:13.948971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:57:13.948971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:57:13.948971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 13:57:13.948971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:57:13.948971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 645/1298] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 11 13:57:13 2022] GET /identity/v3/services/placement => generated 90 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:57:13.953956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96a59b-2769-46e6-863f-dca99696e5b3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:13.972483 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:14.056888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab96a59b-2769-46e6-863f-dca99696e5b3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:14.091403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408622a1-de43-457b-bbcd-0cccc3e73356 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:14.092071 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408622a1-de43-457b-bbcd-0cccc3e73356 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:14.092358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408622a1-de43-457b-bbcd-0cccc3e73356 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:14.092638 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408622a1-de43-457b-bbcd-0cccc3e73356 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:14.093351 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408622a1-de43-457b-bbcd-0cccc3e73356 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:57:14.093662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408622a1-de43-457b-bbcd-0cccc3e73356 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:14.095054 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408622a1-de43-457b-bbcd-0cccc3e73356 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:14.100166 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 654/1299] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Jul 11 13:57:13 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:57:14.106026 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c6316-e51e-4fb1-85e5-180555df23ef None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:14.138521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:14.205932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e09c6316-e51e-4fb1-85e5-180555df23ef None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:14.237387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca65ec8-eff7-42e4-9989-895e51848e2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:14.238026 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca65ec8-eff7-42e4-9989-895e51848e2b None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:14.238351 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca65ec8-eff7-42e4-9989-895e51848e2b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:14.238606 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca65ec8-eff7-42e4-9989-895e51848e2b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:14.239421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca65ec8-eff7-42e4-9989-895e51848e2b None admin] RBAC: Adding query filter params (interface=public, service_id=4ea2c0d492604754a529d37986da732e, region_id=RegionOne) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:57:14.239723 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca65ec8-eff7-42e4-9989-895e51848e2b None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:14.241089 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca65ec8-eff7-42e4-9989-895e51848e2b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:14.246531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 646/1300] 10.222.0.49 () {56 vars in 1251 bytes} [Mon Jul 11 13:57:14 2022] GET /identity/v3/endpoints?service_id=4ea2c0d492604754a529d37986da732e&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:57:14.252377 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408622a1-de43-457b-bbcd-0cccc3e73356 None admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:14.332239 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:14.350240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408622a1-de43-457b-bbcd-0cccc3e73356 None admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:14.384585 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c281002-f20a-4287-a4df-accf3f48f15d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:14.385293 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c281002-f20a-4287-a4df-accf3f48f15d None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:14.385575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c281002-f20a-4287-a4df-accf3f48f15d None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:14.385848 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c281002-f20a-4287-a4df-accf3f48f15d None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:14.386648 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c281002-f20a-4287-a4df-accf3f48f15d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:14.388029 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c281002-f20a-4287-a4df-accf3f48f15d None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:14.395033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 655/1301] 10.222.0.49 () {56 vars in 1087 bytes} [Mon Jul 11 13:57:14 2022] GET /identity/v3/services => generated 2191 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 13:57:16.411768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e418e-b1e7-43f2-94e4-c91ba7cbea1a None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:16.412337 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e418e-b1e7-43f2-94e4-c91ba7cbea1a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:16.412337 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-784e418e-b1e7-43f2-94e4-c91ba7cbea1a None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:16.413776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 647/1302] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:57:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:57:16.420879 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f78b7-6d99-413f-a41d-fd32c1b9c8fc None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:16.421404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f78b7-6d99-413f-a41d-fd32c1b9c8fc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:16.421901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6f78b7-6d99-413f-a41d-fd32c1b9c8fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:16.440486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1d6f78b7-6d99-413f-a41d-fd32c1b9c8fc None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:57:16.441826 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1d6f78b7-6d99-413f-a41d-fd32c1b9c8fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:57:16.514155 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 656/1303] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:57:16 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 11 13:57:16.537345 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e418e-b1e7-43f2-94e4-c91ba7cbea1a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:16.641651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:16.651893 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-784e418e-b1e7-43f2-94e4-c91ba7cbea1a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:16.687235 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f6ac4-bb59-40e9-bd4d-5783e6012263 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:16.687953 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f6ac4-bb59-40e9-bd4d-5783e6012263 None admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:16.688241 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f6ac4-bb59-40e9-bd4d-5783e6012263 None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:16.688520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-507f6ac4-bb59-40e9-bd4d-5783e6012263 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:16.689186 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f6ac4-bb59-40e9-bd4d-5783e6012263 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:16.690609 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-507f6ac4-bb59-40e9-bd4d-5783e6012263 None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:16.703375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application [None req-507f6ac4-bb59-40e9-bd4d-5783e6012263 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 13:57:16.703375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:57:16.703375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 11 13:57:16.703375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 11 13:57:16.703375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 11 13:57:16.703375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 11 13:57:16.703375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 11 13:57:16.703375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:57:16.703375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 13:57:16.703375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:57:16.703375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 11 13:57:16.703375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:57:16.703375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 11 13:57:16.703375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 11 13:57:16.703375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 11 13:57:16.703375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 11 13:57:16.703375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 11 13:57:16.704655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 11 13:57:16.704655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 11 13:57:16.704655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 11 13:57:16.704655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 11 13:57:16.704655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 11 13:57:16.704655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 11 13:57:16.704655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 11 13:57:16.704655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 11 13:57:16.704655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 11 13:57:16.704655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 11 13:57:16.704655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 11 13:57:16.704655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 11 13:57:16.704655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 11 13:57:16.704655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 11 13:57:16.704655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 11 13:57:16.704655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.get_or_create( Jul 11 13:57:16.705727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 11 13:57:16.705727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application with Lock( Jul 11 13:57:16.705727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 11 13:57:16.705727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self._enter() Jul 11 13:57:16.705727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 11 13:57:16.705727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 11 13:57:16.705727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 11 13:57:16.705727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application return self.creator() Jul 11 13:57:16.705727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 11 13:57:16.705727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application created_value = creator( Jul 11 13:57:16.705727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 11 13:57:16.705727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 11 13:57:16.705727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 11 13:57:16.705727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: ERROR keystone.server.flask.application Jul 11 13:57:16.705727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 648/1304] 10.222.0.49 () {56 vars in 1117 bytes} [Mon Jul 11 13:57:16 2022] GET /identity/v3/services/placement => generated 90 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 11 13:57:16.711023 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f78b7-6d99-413f-a41d-fd32c1b9c8fc None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:16.799500 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:16.824783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6f78b7-6d99-413f-a41d-fd32c1b9c8fc None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:16.859804 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b74c2206-d7ce-4574-b8ec-a561ca5335e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:16.860469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c2206-d7ce-4574-b8ec-a561ca5335e9 None admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:16.860752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c2206-d7ce-4574-b8ec-a561ca5335e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:16.861026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b74c2206-d7ce-4574-b8ec-a561ca5335e9 None admin] PATH_INFO: `/v3/services` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:16.861743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c2206-d7ce-4574-b8ec-a561ca5335e9 None admin] RBAC: Adding query filter params (name=placement) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 13:57:16.862045 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c2206-d7ce-4574-b8ec-a561ca5335e9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:16.863699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b74c2206-d7ce-4574-b8ec-a561ca5335e9 None admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:16.868818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 657/1305] 10.222.0.49 () {56 vars in 1116 bytes} [Mon Jul 11 13:57:16 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 13:57:16.875283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f6ac4-bb59-40e9-bd4d-5783e6012263 None admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:57:16.984651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-507f6ac4-bb59-40e9-bd4d-5783e6012263 None admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:57:17.021048 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863669de-9c30-488d-8b13-e0350561684b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:57:17.021717 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863669de-9c30-488d-8b13-e0350561684b None admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:57:17.022018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863669de-9c30-488d-8b13-e0350561684b None admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:57:17.022289 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863669de-9c30-488d-8b13-e0350561684b None admin] PATH_INFO: `/v3/endpoints` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:57:17.023082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863669de-9c30-488d-8b13-e0350561684b None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:57:17.024459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863669de-9c30-488d-8b13-e0350561684b None admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:57:17.050362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 649/1306] 10.222.0.49 () {60 vars in 1144 bytes} [Mon Jul 11 13:57:16 2022] POST /identity/v3/endpoints => generated 331 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 13:57:26.865519 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=30 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:57:27.040767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:20.571598 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e403f-ef71-420f-aca8-727c339331b7 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:20.572772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e403f-ef71-420f-aca8-727c339331b7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:20.573273 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-194e403f-ef71-420f-aca8-727c339331b7 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:20.575666 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 658/1307] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:58:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:58:20.581310 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b42c2b-d312-4d53-9191-a7ff9a826108 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:20.582297 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b42c2b-d312-4d53-9191-a7ff9a826108 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:20.582839 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b42c2b-d312-4d53-9191-a7ff9a826108 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:20.669069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-22b42c2b-d312-4d53-9191-a7ff9a826108 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:58:20.670334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-22b42c2b-d312-4d53-9191-a7ff9a826108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:20.808856 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 650/1308] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:58:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 229 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:58:20.859928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9400c0-f7bb-4f4a-8135-d88585ef017a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:20.860107 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9400c0-f7bb-4f4a-8135-d88585ef017a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:20.860389 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb9400c0-f7bb-4f4a-8135-d88585ef017a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:20.862436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 659/1309] 10.222.0.49 () {54 vars in 964 bytes} [Mon Jul 11 13:58:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:58:20.877397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46350286-8310-4fb2-8b00-a7aed7a005ce None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:20.877702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46350286-8310-4fb2-8b00-a7aed7a005ce None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:20.878004 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46350286-8310-4fb2-8b00-a7aed7a005ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:20.965280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-46350286-8310-4fb2-8b00-a7aed7a005ce None None] MFA Rules not processed for user `ce1c85ee44ef45d2b631e51112b5ea0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:58:20.966563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-46350286-8310-4fb2-8b00-a7aed7a005ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:21.013030 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:21.089118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 651/1310] 10.222.0.49 () {58 vars in 1061 bytes} [Mon Jul 11 13:58:20 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:58:21.098201 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9400c0-f7bb-4f4a-8135-d88585ef017a None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:58:21.100038 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fb9400c0-f7bb-4f4a-8135-d88585ef017a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:21.243339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb9400c0-f7bb-4f4a-8135-d88585ef017a None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:58:21.336493 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9182b581-ed68-4bab-8974-84b9cfe08321 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:58:21.337164 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182b581-ed68-4bab-8974-84b9cfe08321 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:21.337451 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182b581-ed68-4bab-8974-84b9cfe08321 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:21.337736 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9182b581-ed68-4bab-8974-84b9cfe08321 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:21.339469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9182b581-ed68-4bab-8974-84b9cfe08321 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:21.413163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:21.481468 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182b581-ed68-4bab-8974-84b9cfe08321 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:58:21.482956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9182b581-ed68-4bab-8974-84b9cfe08321 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:58:21.507016 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 660/1311] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:58:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:58:23.934207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e73701-934d-42c6-96fc-1af997ee729f None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:23.934767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e73701-934d-42c6-96fc-1af997ee729f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:23.934849 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e73701-934d-42c6-96fc-1af997ee729f None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:23.936284 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 652/1312] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:58:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:58:23.942277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b5c96-68fd-49cb-9fc5-e020cc836325 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:23.943106 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b5c96-68fd-49cb-9fc5-e020cc836325 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:23.943106 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a1b5c96-68fd-49cb-9fc5-e020cc836325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:24.051438 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5a1b5c96-68fd-49cb-9fc5-e020cc836325 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:58:24.053283 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5a1b5c96-68fd-49cb-9fc5-e020cc836325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:24.202532 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 661/1313] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:58:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 258 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:58:24.237918 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e73701-934d-42c6-96fc-1af997ee729f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:58:24.239522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-86e73701-934d-42c6-96fc-1af997ee729f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:24.374255 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e73701-934d-42c6-96fc-1af997ee729f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:58:24.450502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:24.453455 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97bd17-c4a3-43dc-b8d3-5a40716e9bb6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:58:24.454416 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97bd17-c4a3-43dc-b8d3-5a40716e9bb6 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:24.454888 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97bd17-c4a3-43dc-b8d3-5a40716e9bb6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:24.455350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97bd17-c4a3-43dc-b8d3-5a40716e9bb6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:24.457066 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8b97bd17-c4a3-43dc-b8d3-5a40716e9bb6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:24.573561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97bd17-c4a3-43dc-b8d3-5a40716e9bb6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:58:24.575051 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97bd17-c4a3-43dc-b8d3-5a40716e9bb6 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:58:24.593735 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 653/1314] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:58:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:58:26.942013 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43e2b9-541b-4873-8d18-8c7ac4e63e94 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:26.942575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43e2b9-541b-4873-8d18-8c7ac4e63e94 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:26.942697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed43e2b9-541b-4873-8d18-8c7ac4e63e94 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:26.944001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 662/1315] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:58:26.949703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de3d14-18ef-457e-b93f-ab4b702d69d2 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:26.950119 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de3d14-18ef-457e-b93f-ab4b702d69d2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:26.950272 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92de3d14-18ef-457e-b93f-ab4b702d69d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:27.014382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-92de3d14-18ef-457e-b93f-ab4b702d69d2 None None] MFA Rules not processed for user `d820d6d390fa4ca188ab16ae1466c1df`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:58:27.015618 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-92de3d14-18ef-457e-b93f-ab4b702d69d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:27.069111 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:27.113710 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 654/1316] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:58:26 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:58:27.153750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43e2b9-541b-4873-8d18-8c7ac4e63e94 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:58:27.155348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ed43e2b9-541b-4873-8d18-8c7ac4e63e94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:27.165027 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:27.266509 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed43e2b9-541b-4873-8d18-8c7ac4e63e94 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:58:27.342166 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93a5e417-91e2-4ea1-900e-c0392ffa91f8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:58:27.342839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5e417-91e2-4ea1-900e-c0392ffa91f8 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:27.343127 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5e417-91e2-4ea1-900e-c0392ffa91f8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:27.343404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93a5e417-91e2-4ea1-900e-c0392ffa91f8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:27.345063 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-93a5e417-91e2-4ea1-900e-c0392ffa91f8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:27.408210 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:27.459487 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5e417-91e2-4ea1-900e-c0392ffa91f8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:58:27.460998 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93a5e417-91e2-4ea1-900e-c0392ffa91f8 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:58:27.481872 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 663/1317] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:58:27 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:58:30.119404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca59e97-ec8c-44d3-877e-6f62d35e4c23 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:30.120002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca59e97-ec8c-44d3-877e-6f62d35e4c23 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:30.120002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dca59e97-ec8c-44d3-877e-6f62d35e4c23 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:30.121405 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 655/1318] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:58:30.126845 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff9f6c-f42d-4062-874d-9b2e48573da0 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:30.127313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff9f6c-f42d-4062-874d-9b2e48573da0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:30.127409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff9f6c-f42d-4062-874d-9b2e48573da0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:30.194296 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f8ff9f6c-f42d-4062-874d-9b2e48573da0 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:58:30.196123 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f8ff9f6c-f42d-4062-874d-9b2e48573da0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:30.295249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 664/1319] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:58:30 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 13:58:30.335223 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca59e97-ec8c-44d3-877e-6f62d35e4c23 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:58:30.337292 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dca59e97-ec8c-44d3-877e-6f62d35e4c23 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:30.451861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dca59e97-ec8c-44d3-877e-6f62d35e4c23 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:58:30.527719 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a208da-61e5-4c7a-a9e5-516d2caded27 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:58:30.528388 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a208da-61e5-4c7a-a9e5-516d2caded27 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:30.528674 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a208da-61e5-4c7a-a9e5-516d2caded27 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:30.528951 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a208da-61e5-4c7a-a9e5-516d2caded27 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:30.530755 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d5a208da-61e5-4c7a-a9e5-516d2caded27 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:30.539334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:30.645376 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a208da-61e5-4c7a-a9e5-516d2caded27 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:58:30.646862 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a208da-61e5-4c7a-a9e5-516d2caded27 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:58:30.665978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 656/1320] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:58:30 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:58:33.449956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87389198-fa38-4c7d-b620-2261f367e068 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:33.450544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87389198-fa38-4c7d-b620-2261f367e068 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:33.450544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87389198-fa38-4c7d-b620-2261f367e068 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:33.451934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 665/1321] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:58:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:58:33.457204 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ed31e-c702-4e7e-94e8-86fd697675e7 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:33.457627 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ed31e-c702-4e7e-94e8-86fd697675e7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:33.457899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c7ed31e-c702-4e7e-94e8-86fd697675e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:33.523348 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4c7ed31e-c702-4e7e-94e8-86fd697675e7 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:58:33.524575 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4c7ed31e-c702-4e7e-94e8-86fd697675e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:33.583289 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:33.627301 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 657/1322] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:58:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:58:33.664671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87389198-fa38-4c7d-b620-2261f367e068 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:58:33.666261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-87389198-fa38-4c7d-b620-2261f367e068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:33.687195 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:33.780721 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87389198-fa38-4c7d-b620-2261f367e068 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:58:33.861980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763318a3-7d23-404f-906e-b5303959053f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:58:33.862667 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763318a3-7d23-404f-906e-b5303959053f service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:33.862956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763318a3-7d23-404f-906e-b5303959053f service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:33.863236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763318a3-7d23-404f-906e-b5303959053f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:33.864904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-763318a3-7d23-404f-906e-b5303959053f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:33.939541 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:33.981039 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763318a3-7d23-404f-906e-b5303959053f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:58:33.982555 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763318a3-7d23-404f-906e-b5303959053f service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:58:34.002045 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 666/1323] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:58:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:58:36.108088 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf474c-cf5a-4cee-b64e-1daef8fcda97 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:36.108624 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf474c-cf5a-4cee-b64e-1daef8fcda97 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:36.108624 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bf474c-cf5a-4cee-b64e-1daef8fcda97 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:36.109916 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 658/1324] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:58:36.115203 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d17e3-bb8f-400f-b86e-85d0c0c4cfaf None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:36.115644 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d17e3-bb8f-400f-b86e-85d0c0c4cfaf None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:36.115892 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e0d17e3-bb8f-400f-b86e-85d0c0c4cfaf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:36.180926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3e0d17e3-bb8f-400f-b86e-85d0c0c4cfaf None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:58:36.182144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3e0d17e3-bb8f-400f-b86e-85d0c0c4cfaf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:36.279374 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 667/1325] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:58:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:58:36.314977 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf474c-cf5a-4cee-b64e-1daef8fcda97 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:58:36.316541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-10bf474c-cf5a-4cee-b64e-1daef8fcda97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:36.424853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bf474c-cf5a-4cee-b64e-1daef8fcda97 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:58:36.500674 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebab572b-080c-4de1-ab32-fd78e4d53fac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:58:36.501375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab572b-080c-4de1-ab32-fd78e4d53fac service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:36.501657 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab572b-080c-4de1-ab32-fd78e4d53fac service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:36.501930 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebab572b-080c-4de1-ab32-fd78e4d53fac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:36.503648 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ebab572b-080c-4de1-ab32-fd78e4d53fac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:36.523576 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:36.620273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab572b-080c-4de1-ab32-fd78e4d53fac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:58:36.621731 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebab572b-080c-4de1-ab32-fd78e4d53fac service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:58:36.640793 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 659/1326] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:58:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:58:38.729469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1709a38-ac8b-4a69-a01b-233e4b57e486 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:38.729997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1709a38-ac8b-4a69-a01b-233e4b57e486 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:38.730116 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1709a38-ac8b-4a69-a01b-233e4b57e486 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:38.731456 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 668/1327] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:58:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:58:38.736598 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfbfa0-4c21-41c1-9680-ae98a69b494c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:38.737009 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfbfa0-4c21-41c1-9680-ae98a69b494c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:38.737169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bfbfa0-4c21-41c1-9680-ae98a69b494c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:38.801840 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-f5bfbfa0-4c21-41c1-9680-ae98a69b494c None None] MFA Rules not processed for user `d820d6d390fa4ca188ab16ae1466c1df`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:58:38.803095 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f5bfbfa0-4c21-41c1-9680-ae98a69b494c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:38.871052 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:38.906986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 660/1328] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:58:38 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:58:38.945319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1709a38-ac8b-4a69-a01b-233e4b57e486 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:58:38.946954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d1709a38-ac8b-4a69-a01b-233e4b57e486 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:38.978544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:39.052993 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1709a38-ac8b-4a69-a01b-233e4b57e486 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:58:39.133115 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e611c7cf-1dbe-4118-8355-f1d3cda5303d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:58:39.133781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611c7cf-1dbe-4118-8355-f1d3cda5303d service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:39.134072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611c7cf-1dbe-4118-8355-f1d3cda5303d service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:39.134382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e611c7cf-1dbe-4118-8355-f1d3cda5303d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:39.136080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e611c7cf-1dbe-4118-8355-f1d3cda5303d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:39.216150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:39.248427 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611c7cf-1dbe-4118-8355-f1d3cda5303d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:58:39.249887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e611c7cf-1dbe-4118-8355-f1d3cda5303d service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:58:39.268078 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 669/1329] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:58:38 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:58:41.607470 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2800e-4c10-4a47-be58-4c135983a231 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:41.607984 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2800e-4c10-4a47-be58-4c135983a231 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:41.607984 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a2800e-4c10-4a47-be58-4c135983a231 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:41.609340 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 661/1330] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:58:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:58:41.614619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d82aa-25f3-4275-932b-70b51cd71ebb None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:41.615056 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d82aa-25f3-4275-932b-70b51cd71ebb None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:41.615178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-015d82aa-25f3-4275-932b-70b51cd71ebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:41.680589 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-015d82aa-25f3-4275-932b-70b51cd71ebb None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:58:41.682190 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-015d82aa-25f3-4275-932b-70b51cd71ebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:41.782660 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 670/1331] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:58:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:58:41.819387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2800e-4c10-4a47-be58-4c135983a231 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:58:41.820937 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-74a2800e-4c10-4a47-be58-4c135983a231 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:41.928403 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a2800e-4c10-4a47-be58-4c135983a231 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:58:42.000775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe7c83c-1b54-4519-9c60-51cc71349d06 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:58:42.001421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7c83c-1b54-4519-9c60-51cc71349d06 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:42.001703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7c83c-1b54-4519-9c60-51cc71349d06 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:42.001980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe7c83c-1b54-4519-9c60-51cc71349d06 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:42.003655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ebe7c83c-1b54-4519-9c60-51cc71349d06 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:42.030960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:42.112735 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7c83c-1b54-4519-9c60-51cc71349d06 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:58:42.114171 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebe7c83c-1b54-4519-9c60-51cc71349d06 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:58:42.132566 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 662/1332] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:58:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:58:44.208835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef10eac-6535-4054-af81-392267657330 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:44.209408 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef10eac-6535-4054-af81-392267657330 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:44.209408 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ef10eac-6535-4054-af81-392267657330 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:44.211355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 671/1333] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:58:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:58:44.216459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d346156-c447-4fe7-b0e5-f6ab79e8b32b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:44.216886 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d346156-c447-4fe7-b0e5-f6ab79e8b32b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:44.217032 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d346156-c447-4fe7-b0e5-f6ab79e8b32b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:44.280741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-0d346156-c447-4fe7-b0e5-f6ab79e8b32b None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:58:44.281949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0d346156-c447-4fe7-b0e5-f6ab79e8b32b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:44.359976 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:44.383333 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 663/1334] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:58:44.423824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef10eac-6535-4054-af81-392267657330 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:58:44.425380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3ef10eac-6535-4054-af81-392267657330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:44.468356 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:44.535823 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ef10eac-6535-4054-af81-392267657330 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:58:44.610738 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-febbf269-298e-4a4e-b15c-cdec17fa1ee8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:58:44.611333 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbf269-298e-4a4e-b15c-cdec17fa1ee8 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:44.611616 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbf269-298e-4a4e-b15c-cdec17fa1ee8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:44.611893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-febbf269-298e-4a4e-b15c-cdec17fa1ee8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:44.613545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-febbf269-298e-4a4e-b15c-cdec17fa1ee8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:44.717845 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:44.744417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febbf269-298e-4a4e-b15c-cdec17fa1ee8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:58:44.745866 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-febbf269-298e-4a4e-b15c-cdec17fa1ee8 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:58:44.766367 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 672/1335] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:58:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:58:47.171918 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9caca-ed8a-426f-afd7-9b20e09a0f94 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:47.172461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9caca-ed8a-426f-afd7-9b20e09a0f94 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:47.172461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8c9caca-ed8a-426f-afd7-9b20e09a0f94 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:47.173863 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 664/1336] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:58:47.179965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd756b-1c76-4908-85b1-703a67dceb91 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:47.180395 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd756b-1c76-4908-85b1-703a67dceb91 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:47.180531 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebfd756b-1c76-4908-85b1-703a67dceb91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:47.243746 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ebfd756b-1c76-4908-85b1-703a67dceb91 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:58:47.244967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ebfd756b-1c76-4908-85b1-703a67dceb91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:47.340438 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 673/1337] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:58:47 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:58:47.390803 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5ca7f-7198-4769-99bf-483d4749a921 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:47.391095 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5ca7f-7198-4769-99bf-483d4749a921 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:47.391381 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf5ca7f-7198-4769-99bf-483d4749a921 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:47.392796 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 665/1338] 10.222.0.49 () {54 vars in 964 bytes} [Mon Jul 11 13:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:58:47.399193 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404f997-7177-49ad-b7fd-843704636ec0 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:47.399484 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404f997-7177-49ad-b7fd-843704636ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:47.399770 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e404f997-7177-49ad-b7fd-843704636ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:47.453013 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:47.476016 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e404f997-7177-49ad-b7fd-843704636ec0 None None] MFA Rules not processed for user `ce1c85ee44ef45d2b631e51112b5ea0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:58:47.477238 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e404f997-7177-49ad-b7fd-843704636ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:47.556257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 674/1339] 10.222.0.49 () {58 vars in 1061 bytes} [Mon Jul 11 13:58:47 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:58:47.565254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5ca7f-7198-4769-99bf-483d4749a921 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:58:47.566832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0bf5ca7f-7198-4769-99bf-483d4749a921 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:47.669384 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf5ca7f-7198-4769-99bf-483d4749a921 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:58:47.742851 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9e83f1-1278-4e1f-bbd0-7b51870b5028 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:58:47.743496 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e83f1-1278-4e1f-bbd0-7b51870b5028 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:47.743782 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e83f1-1278-4e1f-bbd0-7b51870b5028 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:47.744061 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9e83f1-1278-4e1f-bbd0-7b51870b5028 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:47.745724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ea9e83f1-1278-4e1f-bbd0-7b51870b5028 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:47.783257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:47.857946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e83f1-1278-4e1f-bbd0-7b51870b5028 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:58:47.859405 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea9e83f1-1278-4e1f-bbd0-7b51870b5028 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:58:47.877225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 666/1340] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:58:47 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 312 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:58:52.413162 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3e4b4-bbf7-4733-bdd0-59bcecedf710 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:52.413673 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3e4b4-bbf7-4733-bdd0-59bcecedf710 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:52.413788 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db3e4b4-bbf7-4733-bdd0-59bcecedf710 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:52.415139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 675/1341] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:58:52.420604 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e627e-9fab-4f25-95a6-083dae7176ac None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:52.421001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e627e-9fab-4f25-95a6-083dae7176ac None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:52.421344 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-792e627e-9fab-4f25-95a6-083dae7176ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:52.490112 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-792e627e-9fab-4f25-95a6-083dae7176ac None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:58:52.491973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-792e627e-9fab-4f25-95a6-083dae7176ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:52.578093 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:52.589589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 667/1342] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:58:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 13:58:52.627872 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3e4b4-bbf7-4733-bdd0-59bcecedf710 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:58:52.629396 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1db3e4b4-bbf7-4733-bdd0-59bcecedf710 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:52.734200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db3e4b4-bbf7-4733-bdd0-59bcecedf710 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:58:52.778993 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:52.820151 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729d8924-0afa-485c-aefe-dfb279c90703 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:58:52.820809 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d8924-0afa-485c-aefe-dfb279c90703 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:52.821092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d8924-0afa-485c-aefe-dfb279c90703 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:52.821371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729d8924-0afa-485c-aefe-dfb279c90703 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:52.823075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-729d8924-0afa-485c-aefe-dfb279c90703 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:52.935832 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d8924-0afa-485c-aefe-dfb279c90703 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:58:52.937287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729d8924-0afa-485c-aefe-dfb279c90703 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:58:52.956628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 676/1343] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:58:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:58:55.689288 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c04e0-531c-43f4-a853-5b4c68796582 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:55.689846 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c04e0-531c-43f4-a853-5b4c68796582 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:55.689846 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf5c04e0-531c-43f4-a853-5b4c68796582 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:55.691333 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 668/1344] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:58:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:58:55.697825 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f851543-99ed-420f-80ea-0c8ab55ea190 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:55.698373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f851543-99ed-420f-80ea-0c8ab55ea190 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:55.698493 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f851543-99ed-420f-80ea-0c8ab55ea190 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:55.771831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6f851543-99ed-420f-80ea-0c8ab55ea190 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:58:55.773173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6f851543-99ed-420f-80ea-0c8ab55ea190 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:55.795132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:55.884387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 677/1345] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:58:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:58:55.922916 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c04e0-531c-43f4-a853-5b4c68796582 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:58:55.924501 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-cf5c04e0-531c-43f4-a853-5b4c68796582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:56.035628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf5c04e0-531c-43f4-a853-5b4c68796582 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:58:56.121729 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c5f8ed7-b6da-4de7-8c07-5118fb62131b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:58:56.122423 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f8ed7-b6da-4de7-8c07-5118fb62131b service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:58:56.122711 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f8ed7-b6da-4de7-8c07-5118fb62131b service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:58:56.122989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c5f8ed7-b6da-4de7-8c07-5118fb62131b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:58:56.124665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2c5f8ed7-b6da-4de7-8c07-5118fb62131b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:58:56.170343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:58:56.237397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f8ed7-b6da-4de7-8c07-5118fb62131b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:58:56.238885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c5f8ed7-b6da-4de7-8c07-5118fb62131b service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:58:56.257250 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 669/1346] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:58:55 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:59:00.263054 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c9fc1-4d93-40d4-a61b-588d96cab44c None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:00.263653 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c9fc1-4d93-40d4-a61b-588d96cab44c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:00.263653 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44c9fc1-4d93-40d4-a61b-588d96cab44c None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:00.264997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 678/1347] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:00.270335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacf9be-6aff-4753-9c96-330a11977f98 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:00.270783 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacf9be-6aff-4753-9c96-330a11977f98 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:00.270885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cacf9be-6aff-4753-9c96-330a11977f98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:00.337066 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9cacf9be-6aff-4753-9c96-330a11977f98 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:00.338294 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9cacf9be-6aff-4753-9c96-330a11977f98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:00.448071 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:00.454371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 670/1348] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:59:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:00.489445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c9fc1-4d93-40d4-a61b-588d96cab44c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:59:00.491655 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e44c9fc1-4d93-40d4-a61b-588d96cab44c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:00.620129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44c9fc1-4d93-40d4-a61b-588d96cab44c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:59:00.677131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:00.706183 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4c6c4f-089d-480a-afa8-4606eecb4d04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:59:00.706913 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c6c4f-089d-480a-afa8-4606eecb4d04 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:00.707197 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c6c4f-089d-480a-afa8-4606eecb4d04 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:00.707476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4c6c4f-089d-480a-afa8-4606eecb4d04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:00.709692 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dd4c6c4f-089d-480a-afa8-4606eecb4d04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:00.843140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c6c4f-089d-480a-afa8-4606eecb4d04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:59:00.844679 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4c6c4f-089d-480a-afa8-4606eecb4d04 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:59:00.866039 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 679/1349] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:59:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 13:59:05.782474 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddd48b-2302-4ec3-9ad4-8526d185a90a None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:05.783338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddd48b-2302-4ec3-9ad4-8526d185a90a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:05.783338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddd48b-2302-4ec3-9ad4-8526d185a90a None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:05.784663 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 671/1350] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 11 13:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:05.789718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a2a43-75e6-46c6-87d4-0cdda72ace3b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:05.790114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a2a43-75e6-46c6-87d4-0cdda72ace3b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:05.790282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e4a2a43-75e6-46c6-87d4-0cdda72ace3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:05.936359 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8e4a2a43-75e6-46c6-87d4-0cdda72ace3b None None] MFA Rules not processed for user `ee29b7f1cea64d6781e1df5c8021363f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:05.938144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8e4a2a43-75e6-46c6-87d4-0cdda72ace3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:05.961510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:06.076703 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 680/1351] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 11 13:59:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 288 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:10.238355 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb618b-077e-46c6-947c-393b73da254a None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:10.238924 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb618b-077e-46c6-947c-393b73da254a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:10.238924 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08bb618b-077e-46c6-947c-393b73da254a None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:10.240225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 672/1352] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 11 13:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:10.246849 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347cf11-9c65-47f1-a43d-c4b7ee78ffe6 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:10.247286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347cf11-9c65-47f1-a43d-c4b7ee78ffe6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:10.247392 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347cf11-9c65-47f1-a43d-c4b7ee78ffe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:10.327096 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3347cf11-9c65-47f1-a43d-c4b7ee78ffe6 None None] MFA Rules not processed for user `ee29b7f1cea64d6781e1df5c8021363f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:10.328241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3347cf11-9c65-47f1-a43d-c4b7ee78ffe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:10.405108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:10.445900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 681/1353] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 11 13:59:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:11.836950 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8b5af-fb2a-45a2-9fb1-2551ce30735d None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:11.837499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8b5af-fb2a-45a2-9fb1-2551ce30735d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:11.837499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2aa8b5af-fb2a-45a2-9fb1-2551ce30735d None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:11.838982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 673/1354] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 11 13:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:11.844401 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dfa03-41b8-4dbf-8fc2-28752e8bc6d4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:11.844889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dfa03-41b8-4dbf-8fc2-28752e8bc6d4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:11.844984 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1dfa03-41b8-4dbf-8fc2-28752e8bc6d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:11.937232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0d1dfa03-41b8-4dbf-8fc2-28752e8bc6d4 None None] MFA Rules not processed for user `ee29b7f1cea64d6781e1df5c8021363f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:11.938508 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0d1dfa03-41b8-4dbf-8fc2-28752e8bc6d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:12.048126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 682/1355] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 11 13:59:11 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:15.443492 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f308c98-f017-4f3c-af55-551eece5757c None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:15.444625 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f308c98-f017-4f3c-af55-551eece5757c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:15.445109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f308c98-f017-4f3c-af55-551eece5757c None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:15.448811 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 674/1356] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:59:15 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:15.456127 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73597e-358f-4510-afe9-617e1ba68530 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:15.456967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73597e-358f-4510-afe9-617e1ba68530 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:15.457438 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af73597e-358f-4510-afe9-617e1ba68530 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:15.508016 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:15.547697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-af73597e-358f-4510-afe9-617e1ba68530 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:15.548898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-af73597e-358f-4510-afe9-617e1ba68530 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:15.673047 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 683/1357] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:59:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:15.736533 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33f1fc-3a6f-4fd7-b00b-49fa544cba72 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:15.737713 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33f1fc-3a6f-4fd7-b00b-49fa544cba72 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:15.738220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a33f1fc-3a6f-4fd7-b00b-49fa544cba72 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:15.741895 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 675/1358] 10.222.0.49 () {54 vars in 961 bytes} [Mon Jul 11 13:59:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:15.752874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163e159-cb1c-4e1f-90e5-58d4eb03e438 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:15.753497 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163e159-cb1c-4e1f-90e5-58d4eb03e438 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:15.754006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8163e159-cb1c-4e1f-90e5-58d4eb03e438 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:15.860443 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8163e159-cb1c-4e1f-90e5-58d4eb03e438 None None] MFA Rules not processed for user `ef5dac3f311e43809a46995bab56e780`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:15.861668 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8163e159-cb1c-4e1f-90e5-58d4eb03e438 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:15.930597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:16.047144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 684/1359] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Jul 11 13:59:15 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:16.083899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33f1fc-3a6f-4fd7-b00b-49fa544cba72 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:59:16.083899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6a33f1fc-3a6f-4fd7-b00b-49fa544cba72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:16.274008 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a33f1fc-3a6f-4fd7-b00b-49fa544cba72 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:59:16.393674 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:16.451494 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fdf8ab4-7ff9-4620-91dc-09dee7e79202 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:59:16.452773 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf8ab4-7ff9-4620-91dc-09dee7e79202 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:16.453166 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf8ab4-7ff9-4620-91dc-09dee7e79202 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:16.453801 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fdf8ab4-7ff9-4620-91dc-09dee7e79202 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:16.456167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6fdf8ab4-7ff9-4620-91dc-09dee7e79202 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:16.568385 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf8ab4-7ff9-4620-91dc-09dee7e79202 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:59:16.570290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fdf8ab4-7ff9-4620-91dc-09dee7e79202 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:59:16.592241 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 676/1360] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:59:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 529 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:59:18.885287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f121d1d-1418-4489-a46a-a823becdc4f0 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:18.885287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f121d1d-1418-4489-a46a-a823becdc4f0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:18.887080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f121d1d-1418-4489-a46a-a823becdc4f0 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:18.887080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 685/1361] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:59:18.893349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba44237-d433-4eb3-adb4-b9057b9eb00b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:18.894033 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba44237-d433-4eb3-adb4-b9057b9eb00b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:18.894033 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba44237-d433-4eb3-adb4-b9057b9eb00b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:18.964770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-bba44237-d433-4eb3-adb4-b9057b9eb00b None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:18.966020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bba44237-d433-4eb3-adb4-b9057b9eb00b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:19.005640 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:19.074221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 677/1362] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:59:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:19.122714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f121d1d-1418-4489-a46a-a823becdc4f0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:59:19.124283 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7f121d1d-1418-4489-a46a-a823becdc4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:19.309857 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f121d1d-1418-4489-a46a-a823becdc4f0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:59:19.320583 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:19.473635 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16855db5-16d8-4890-b28a-92af9fe36235 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:59:19.475095 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16855db5-16d8-4890-b28a-92af9fe36235 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:19.476147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16855db5-16d8-4890-b28a-92af9fe36235 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:19.476531 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16855db5-16d8-4890-b28a-92af9fe36235 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:19.478425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-16855db5-16d8-4890-b28a-92af9fe36235 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:19.593833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16855db5-16d8-4890-b28a-92af9fe36235 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:59:19.595761 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16855db5-16d8-4890-b28a-92af9fe36235 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:59:19.618335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 686/1363] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:59:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:59:22.044566 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1b46b-3bb9-4f47-b22a-da0d48abb77c None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:22.045595 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1b46b-3bb9-4f47-b22a-da0d48abb77c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:22.046047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a1b46b-3bb9-4f47-b22a-da0d48abb77c None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:22.047682 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 678/1364] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:59:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:22.056564 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f88bf3-9f06-4513-b210-8b51d5a413ff None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:22.057130 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f88bf3-9f06-4513-b210-8b51d5a413ff None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:22.057130 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f88bf3-9f06-4513-b210-8b51d5a413ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:22.073993 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:22.145614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e9f88bf3-9f06-4513-b210-8b51d5a413ff None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:22.147111 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e9f88bf3-9f06-4513-b210-8b51d5a413ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:22.270367 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 687/1365] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:59:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:22.350392 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efe39b-5927-4fd2-b9df-d1a1bb561395 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:22.350923 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efe39b-5927-4fd2-b9df-d1a1bb561395 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:22.351331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60efe39b-5927-4fd2-b9df-d1a1bb561395 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:22.353158 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 679/1366] 10.222.0.49 () {54 vars in 961 bytes} [Mon Jul 11 13:59:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:22.359361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f96fae-0d78-445e-ac77-55c8e2416acf None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:22.359639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f96fae-0d78-445e-ac77-55c8e2416acf None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:22.359917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45f96fae-0d78-445e-ac77-55c8e2416acf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:22.454627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-45f96fae-0d78-445e-ac77-55c8e2416acf None None] MFA Rules not processed for user `ef5dac3f311e43809a46995bab56e780`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:22.456626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-45f96fae-0d78-445e-ac77-55c8e2416acf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:22.525183 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:22.626789 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 688/1367] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Jul 11 13:59:22 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 268 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:22.635162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efe39b-5927-4fd2-b9df-d1a1bb561395 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:59:22.636973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-60efe39b-5927-4fd2-b9df-d1a1bb561395 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:22.816373 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60efe39b-5927-4fd2-b9df-d1a1bb561395 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:59:22.845400 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:22.970881 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96d1ae4e-440e-48b2-bf89-96a1637890af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:59:22.971705 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1ae4e-440e-48b2-bf89-96a1637890af service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:22.972080 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1ae4e-440e-48b2-bf89-96a1637890af service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:22.972452 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96d1ae4e-440e-48b2-bf89-96a1637890af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:22.974255 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-96d1ae4e-440e-48b2-bf89-96a1637890af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:23.081653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1ae4e-440e-48b2-bf89-96a1637890af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:59:23.087992 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96d1ae4e-440e-48b2-bf89-96a1637890af service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:59:23.106642 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 680/1368] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:59:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 472 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:59:23.858623 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11581024-52a1-4e3c-a5f0-f72f28cb256a None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:23.859246 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11581024-52a1-4e3c-a5f0-f72f28cb256a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:23.859246 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11581024-52a1-4e3c-a5f0-f72f28cb256a None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:23.860870 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 689/1369] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:59:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:59:23.867047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828d5f3-77f5-462d-8465-c632f17e35f0 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:23.867366 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828d5f3-77f5-462d-8465-c632f17e35f0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:23.867614 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9828d5f3-77f5-462d-8465-c632f17e35f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:23.904822 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:23.952526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9828d5f3-77f5-462d-8465-c632f17e35f0 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:23.954270 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9828d5f3-77f5-462d-8465-c632f17e35f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:24.117412 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 681/1370] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:59:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 252 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:24.459060 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975a6f3-18b2-4205-ac4b-21cc7d71c174 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:24.459350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975a6f3-18b2-4205-ac4b-21cc7d71c174 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:24.459632 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d975a6f3-18b2-4205-ac4b-21cc7d71c174 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:24.461465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 690/1371] 10.222.0.49 () {54 vars in 963 bytes} [Mon Jul 11 13:59:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:24.466056 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd7b9d-c127-47b0-8028-03f341a0c065 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:24.466370 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd7b9d-c127-47b0-8028-03f341a0c065 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:24.466719 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd7b9d-c127-47b0-8028-03f341a0c065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:24.543982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-00bd7b9d-c127-47b0-8028-03f341a0c065 None None] MFA Rules not processed for user `6bb1616f632d4c67828cce3fc41945b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:24.545358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-00bd7b9d-c127-47b0-8028-03f341a0c065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:24.605917 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:24.639303 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 682/1372] 10.222.0.49 () {58 vars in 1060 bytes} [Mon Jul 11 13:59:24 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:24.646843 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975a6f3-18b2-4205-ac4b-21cc7d71c174 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:59:24.648467 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d975a6f3-18b2-4205-ac4b-21cc7d71c174 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:24.758252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d975a6f3-18b2-4205-ac4b-21cc7d71c174 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:59:24.817710 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:24.862980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d60b95d-5320-44e9-93f4-564d2bfe023f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:59:24.863747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60b95d-5320-44e9-93f4-564d2bfe023f service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:24.864039 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60b95d-5320-44e9-93f4-564d2bfe023f service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:24.864520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d60b95d-5320-44e9-93f4-564d2bfe023f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:24.865859 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2d60b95d-5320-44e9-93f4-564d2bfe023f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:25.005419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60b95d-5320-44e9-93f4-564d2bfe023f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:59:25.007480 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d60b95d-5320-44e9-93f4-564d2bfe023f service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:59:25.052404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 691/1373] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:59:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:59:25.278618 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf7a77-3ea7-4694-8441-eb1d2a105568 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:25.279145 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf7a77-3ea7-4694-8441-eb1d2a105568 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:25.279145 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6edf7a77-3ea7-4694-8441-eb1d2a105568 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:25.280560 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 683/1374] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:25.290365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f33dbc-9218-4353-b380-d0b831710bc1 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:25.290665 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f33dbc-9218-4353-b380-d0b831710bc1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:25.290955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f33dbc-9218-4353-b380-d0b831710bc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:25.363991 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-59f33dbc-9218-4353-b380-d0b831710bc1 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:25.365855 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-59f33dbc-9218-4353-b380-d0b831710bc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:25.386866 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:25.479252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 692/1375] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:59:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:25.523027 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf7a77-3ea7-4694-8441-eb1d2a105568 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:59:25.524636 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6edf7a77-3ea7-4694-8441-eb1d2a105568 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:25.673790 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6edf7a77-3ea7-4694-8441-eb1d2a105568 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:59:25.731254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:25.814137 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef8b1df9-e754-4156-bf6f-f8086ce21ae0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:59:25.814829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b1df9-e754-4156-bf6f-f8086ce21ae0 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:25.815110 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b1df9-e754-4156-bf6f-f8086ce21ae0 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:25.815384 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef8b1df9-e754-4156-bf6f-f8086ce21ae0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:25.817097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ef8b1df9-e754-4156-bf6f-f8086ce21ae0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:25.927441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b1df9-e754-4156-bf6f-f8086ce21ae0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:59:25.929237 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef8b1df9-e754-4156-bf6f-f8086ce21ae0 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:59:25.948158 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 684/1376] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:59:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 426 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:59:28.056011 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93927b05-32cf-4cc5-8724-2a206c69efd7 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:28.056519 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93927b05-32cf-4cc5-8724-2a206c69efd7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:28.056614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93927b05-32cf-4cc5-8724-2a206c69efd7 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:28.058023 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 693/1377] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:59:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:28.064259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf22d09-c917-483d-a905-c72ccdddedbd None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:28.064743 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf22d09-c917-483d-a905-c72ccdddedbd None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:28.064841 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf22d09-c917-483d-a905-c72ccdddedbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:28.150336 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3bf22d09-c917-483d-a905-c72ccdddedbd None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:28.152314 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3bf22d09-c917-483d-a905-c72ccdddedbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:28.165156 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:28.289006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 685/1378] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:59:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:28.340354 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93927b05-32cf-4cc5-8724-2a206c69efd7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:59:28.341887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-93927b05-32cf-4cc5-8724-2a206c69efd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:28.510962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93927b05-32cf-4cc5-8724-2a206c69efd7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:59:28.514769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:28.704117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f68e851-e2e7-4f8e-ab81-3a4389e13c2e service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:59:28.704525 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68e851-e2e7-4f8e-ab81-3a4389e13c2e service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:28.705454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68e851-e2e7-4f8e-ab81-3a4389e13c2e service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:28.706169 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f68e851-e2e7-4f8e-ab81-3a4389e13c2e service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:28.708618 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8f68e851-e2e7-4f8e-ab81-3a4389e13c2e service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:28.856433 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68e851-e2e7-4f8e-ab81-3a4389e13c2e service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:59:28.858007 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f68e851-e2e7-4f8e-ab81-3a4389e13c2e service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:59:28.880709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 694/1379] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:59:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 541 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:59:31.025121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108e7ab-ab73-4a37-99aa-63ca93a244da None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:31.025709 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108e7ab-ab73-4a37-99aa-63ca93a244da None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:31.025709 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7108e7ab-ab73-4a37-99aa-63ca93a244da None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:31.027325 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 686/1380] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:59:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:31.032806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c944e3d-7437-4f20-80d5-7fa810f76cf7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:31.033307 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c944e3d-7437-4f20-80d5-7fa810f76cf7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:31.033421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c944e3d-7437-4f20-80d5-7fa810f76cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:31.044344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:31.123374 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4c944e3d-7437-4f20-80d5-7fa810f76cf7 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:31.124606 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4c944e3d-7437-4f20-80d5-7fa810f76cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:31.251111 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 695/1381] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:31.282873 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108e7ab-ab73-4a37-99aa-63ca93a244da None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:59:31.284282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7108e7ab-ab73-4a37-99aa-63ca93a244da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:31.442337 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:31.453967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7108e7ab-ab73-4a37-99aa-63ca93a244da None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:59:31.597097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a5b626d-dabb-43a1-8f13-bbcdb468f944 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:59:31.597872 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b626d-dabb-43a1-8f13-bbcdb468f944 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:31.597872 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b626d-dabb-43a1-8f13-bbcdb468f944 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:31.598190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a5b626d-dabb-43a1-8f13-bbcdb468f944 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:31.600617 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4a5b626d-dabb-43a1-8f13-bbcdb468f944 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:31.721588 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b626d-dabb-43a1-8f13-bbcdb468f944 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:59:31.723573 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a5b626d-dabb-43a1-8f13-bbcdb468f944 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:59:31.739166 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:31.757742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 687/1382] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:59:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 475 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:59:33.919143 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f35d17-abc6-4db9-8779-629615ca59c4 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:33.919898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f35d17-abc6-4db9-8779-629615ca59c4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:33.919898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67f35d17-abc6-4db9-8779-629615ca59c4 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:33.921110 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 696/1383] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:59:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:33.927564 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f4d38-a284-4ede-813e-1917f945908b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:33.928066 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f4d38-a284-4ede-813e-1917f945908b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:33.928161 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-281f4d38-a284-4ede-813e-1917f945908b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:34.006444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-281f4d38-a284-4ede-813e-1917f945908b None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:34.007687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-281f4d38-a284-4ede-813e-1917f945908b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:34.124647 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 688/1384] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:59:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:34.167061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f35d17-abc6-4db9-8779-629615ca59c4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:59:34.168765 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-67f35d17-abc6-4db9-8779-629615ca59c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:34.267369 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:34.345826 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67f35d17-abc6-4db9-8779-629615ca59c4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:59:34.480348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebdc6fdd-6723-473a-b885-d403bd2f93bd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:59:34.481017 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc6fdd-6723-473a-b885-d403bd2f93bd service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:34.481330 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc6fdd-6723-473a-b885-d403bd2f93bd service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:34.481583 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebdc6fdd-6723-473a-b885-d403bd2f93bd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:34.483305 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ebdc6fdd-6723-473a-b885-d403bd2f93bd service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:34.537814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:34.586172 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc6fdd-6723-473a-b885-d403bd2f93bd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:59:34.588017 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebdc6fdd-6723-473a-b885-d403bd2f93bd service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:59:34.609241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 697/1385] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:59:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 13:59:36.956534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb91b6-6cb2-4ab9-a730-92413c2d7a4e None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:36.957109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb91b6-6cb2-4ab9-a730-92413c2d7a4e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:36.957109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aedb91b6-6cb2-4ab9-a730-92413c2d7a4e None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:36.958559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 689/1386] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:59:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:36.964908 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83258b-ab96-4c2d-b4d0-81c787fa120e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:36.965490 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83258b-ab96-4c2d-b4d0-81c787fa120e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:36.965763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c83258b-ab96-4c2d-b4d0-81c787fa120e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:37.056565 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4c83258b-ab96-4c2d-b4d0-81c787fa120e None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:37.057838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4c83258b-ab96-4c2d-b4d0-81c787fa120e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:37.187170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 698/1387] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:59:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:37.231947 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb91b6-6cb2-4ab9-a730-92413c2d7a4e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:59:37.233483 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-aedb91b6-6cb2-4ab9-a730-92413c2d7a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:37.313281 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:37.389207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aedb91b6-6cb2-4ab9-a730-92413c2d7a4e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:59:37.531084 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce61554-ca9b-4eee-92ba-b6c7d93870d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:59:37.531897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce61554-ca9b-4eee-92ba-b6c7d93870d6 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:37.532390 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce61554-ca9b-4eee-92ba-b6c7d93870d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:37.532868 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce61554-ca9b-4eee-92ba-b6c7d93870d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:37.534678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dce61554-ca9b-4eee-92ba-b6c7d93870d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:37.578137 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:37.639253 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce61554-ca9b-4eee-92ba-b6c7d93870d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:59:37.640889 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce61554-ca9b-4eee-92ba-b6c7d93870d6 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:59:37.661859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 690/1388] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:59:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:59:39.745518 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df65cd-6caa-403b-b356-b46559fb9af0 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:39.746084 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df65cd-6caa-403b-b356-b46559fb9af0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:39.746084 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60df65cd-6caa-403b-b356-b46559fb9af0 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:39.747485 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 699/1389] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:39.752841 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713b404-910e-4fde-8382-f91205f47419 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:39.753354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713b404-910e-4fde-8382-f91205f47419 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:39.753452 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a713b404-910e-4fde-8382-f91205f47419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:39.823145 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a713b404-910e-4fde-8382-f91205f47419 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:39.824369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a713b404-910e-4fde-8382-f91205f47419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:39.929668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 691/1390] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:59:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:39.975502 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df65cd-6caa-403b-b356-b46559fb9af0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:59:39.977098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-60df65cd-6caa-403b-b356-b46559fb9af0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:40.012537 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:40.144825 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60df65cd-6caa-403b-b356-b46559fb9af0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:59:40.256185 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:40.269569 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c103ebc-8cdf-46b1-a76e-fb764541eec1 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:59:40.270147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c103ebc-8cdf-46b1-a76e-fb764541eec1 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:40.270512 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c103ebc-8cdf-46b1-a76e-fb764541eec1 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:40.270785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c103ebc-8cdf-46b1-a76e-fb764541eec1 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:40.272514 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6c103ebc-8cdf-46b1-a76e-fb764541eec1 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:40.372187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c103ebc-8cdf-46b1-a76e-fb764541eec1 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:59:40.374076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c103ebc-8cdf-46b1-a76e-fb764541eec1 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:59:40.393549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 700/1391] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:59:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:59:40.533284 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdd44d-8bbb-49fc-b277-71eaf4228a10 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:40.533544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdd44d-8bbb-49fc-b277-71eaf4228a10 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:40.533882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74bdd44d-8bbb-49fc-b277-71eaf4228a10 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:40.535458 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 692/1392] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:59:40 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:40.542291 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37525a-4c8d-48cd-ae37-810c0466486a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:40.542645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37525a-4c8d-48cd-ae37-810c0466486a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:40.542935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37525a-4c8d-48cd-ae37-810c0466486a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:40.629387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0f37525a-4c8d-48cd-ae37-810c0466486a None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:40.630902 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0f37525a-4c8d-48cd-ae37-810c0466486a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:40.708280 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:40.753763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 701/1393] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:59:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 213 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:40.812737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09244ad-7378-4d72-827a-f946c41e492f None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:40.813326 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09244ad-7378-4d72-827a-f946c41e492f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:40.813326 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c09244ad-7378-4d72-827a-f946c41e492f None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:40.814730 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 693/1394] 10.222.0.49 () {54 vars in 962 bytes} [Mon Jul 11 13:59:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:40.820375 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b18869-edae-47ed-9b48-657dd49e6f70 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:40.820665 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b18869-edae-47ed-9b48-657dd49e6f70 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:40.820945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62b18869-edae-47ed-9b48-657dd49e6f70 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:40.908594 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-62b18869-edae-47ed-9b48-657dd49e6f70 None None] MFA Rules not processed for user `f80d59fec4994e689e857ba2087d3a23`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:40.909857 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-62b18869-edae-47ed-9b48-657dd49e6f70 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:41.001943 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 702/1395] 10.222.0.49 () {58 vars in 1059 bytes} [Mon Jul 11 13:59:40 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:41.010264 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09244ad-7378-4d72-827a-f946c41e492f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:59:41.011827 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c09244ad-7378-4d72-827a-f946c41e492f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:41.026494 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:41.142174 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c09244ad-7378-4d72-827a-f946c41e492f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:59:41.231600 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b4b129-78f1-4f49-ab14-a6425154b819 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f80d59fec4994e689e857ba2087d3a23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:59:41.232253 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4b129-78f1-4f49-ab14-a6425154b819 service glance] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:41.232538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4b129-78f1-4f49-ab14-a6425154b819 service glance] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:41.232814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b4b129-78f1-4f49-ab14-a6425154b819 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:41.234542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-73b4b129-78f1-4f49-ab14-a6425154b819 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:41.308446 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:41.351909 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b4b129-78f1-4f49-ab14-a6425154b819 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:59:41.353307 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b4b129-78f1-4f49-ab14-a6425154b819 service glance] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:59:41.373127 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 694/1396] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:59:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 363 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:59:42.112015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8feab7-4f35-4f35-b64b-cd1ac0802a45 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:42.112544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8feab7-4f35-4f35-b64b-cd1ac0802a45 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:42.112664 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8feab7-4f35-4f35-b64b-cd1ac0802a45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:42.195515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-af8feab7-4f35-4f35-b64b-cd1ac0802a45 None None] MFA Rules not processed for user `52076f59f68f4d77a0dadbf7a98d2b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:42.196772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-af8feab7-4f35-4f35-b64b-cd1ac0802a45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:42.205443 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:42.289475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 703/1397] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 11 13:59:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:42.336286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04186864-ae50-43e8-8eb0-2a851481f3e2 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:42.336866 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04186864-ae50-43e8-8eb0-2a851481f3e2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:42.336866 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04186864-ae50-43e8-8eb0-2a851481f3e2 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:42.338270 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 695/1398] 10.222.0.49 () {54 vars in 942 bytes} [Mon Jul 11 13:59:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:42.344619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b15a6-b441-4705-91ac-679f916be848 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:42.344916 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b15a6-b441-4705-91ac-679f916be848 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:42.345196 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295b15a6-b441-4705-91ac-679f916be848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:42.422880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-295b15a6-b441-4705-91ac-679f916be848 None None] MFA Rules not processed for user `d9c1369033a44c1c81d4c1c9ff2bbedb`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:42.424088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-295b15a6-b441-4705-91ac-679f916be848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:42.533884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:42.535170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4c0aa-f8b9-4da2-894a-dcb6f706c1e0 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:42.535562 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4c0aa-f8b9-4da2-894a-dcb6f706c1e0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:42.535886 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab4c0aa-f8b9-4da2-894a-dcb6f706c1e0 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:42.537861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 696/1399] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:59:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 13:59:42.543755 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d58d9c-ac6f-425f-886a-f71fbf5cbc04 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:42.544057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d58d9c-ac6f-425f-886a-f71fbf5cbc04 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:42.544360 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19d58d9c-ac6f-425f-886a-f71fbf5cbc04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:42.562512 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 704/1400] 10.222.0.49 () {58 vars in 1039 bytes} [Mon Jul 11 13:59:42 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:42.575712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b15a6-b441-4705-91ac-679f916be848 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:59:42.587052 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295b15a6-b441-4705-91ac-679f916be848 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:59:42.615290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-19d58d9c-ac6f-425f-886a-f71fbf5cbc04 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:42.616556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-19d58d9c-ac6f-425f-886a-f71fbf5cbc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:42.709509 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62d9a0bf-aa69-4642-b788-fc88d82b7f5b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c1369033a44c1c81d4c1c9ff2bbedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:59:42.710308 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9a0bf-aa69-4642-b788-fc88d82b7f5b service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:42.710618 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9a0bf-aa69-4642-b788-fc88d82b7f5b service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:42.710895 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62d9a0bf-aa69-4642-b788-fc88d82b7f5b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:42.712733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-62d9a0bf-aa69-4642-b788-fc88d82b7f5b service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:42.725461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 697/1401] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:59:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 13:59:42.769393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d58d9c-ac6f-425f-886a-f71fbf5cbc04 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:59:42.771216 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-19d58d9c-ac6f-425f-886a-f71fbf5cbc04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:42.798520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:42.806485 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9a0bf-aa69-4642-b788-fc88d82b7f5b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:59:42.808107 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62d9a0bf-aa69-4642-b788-fc88d82b7f5b service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:59:42.816342 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 705/1402] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 11 13:59:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 13:59:42.896874 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19d58d9c-ac6f-425f-886a-f71fbf5cbc04 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:59:43.032493 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead52389-36a0-4871-b906-9fd93ce2cd5c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:59:43.033065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead52389-36a0-4871-b906-9fd93ce2cd5c service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:43.033353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead52389-36a0-4871-b906-9fd93ce2cd5c service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:43.033627 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead52389-36a0-4871-b906-9fd93ce2cd5c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:43.035323 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ead52389-36a0-4871-b906-9fd93ce2cd5c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:43.055658 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:43.145668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead52389-36a0-4871-b906-9fd93ce2cd5c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:59:43.147308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ead52389-36a0-4871-b906-9fd93ce2cd5c service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:59:43.166316 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 698/1403] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:59:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 13:59:45.345494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78b3fd-d2f0-4719-a9c8-1f2b7cd010b7 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:45.346071 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78b3fd-d2f0-4719-a9c8-1f2b7cd010b7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:45.346071 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f78b3fd-d2f0-4719-a9c8-1f2b7cd010b7 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:45.347861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 706/1404] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:59:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:45.354586 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a711a21-92c0-4d35-a663-6fe6d65ab914 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:45.355157 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a711a21-92c0-4d35-a663-6fe6d65ab914 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:45.355157 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a711a21-92c0-4d35-a663-6fe6d65ab914 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:45.435515 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-7a711a21-92c0-4d35-a663-6fe6d65ab914 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:45.437119 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7a711a21-92c0-4d35-a663-6fe6d65ab914 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:45.559093 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:45.570431 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 699/1405] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:59:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 13:59:45.617310 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78b3fd-d2f0-4719-a9c8-1f2b7cd010b7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:59:45.617310 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6f78b3fd-d2f0-4719-a9c8-1f2b7cd010b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:45.628591 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:45.806288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f78b3fd-d2f0-4719-a9c8-1f2b7cd010b7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:59:45.900821 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:45.939315 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16931716-db71-4fb5-b115-c2c969f6c2d6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:59:45.940548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16931716-db71-4fb5-b115-c2c969f6c2d6 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:45.940831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16931716-db71-4fb5-b115-c2c969f6c2d6 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:45.941106 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16931716-db71-4fb5-b115-c2c969f6c2d6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:45.942869 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-16931716-db71-4fb5-b115-c2c969f6c2d6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:46.054282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16931716-db71-4fb5-b115-c2c969f6c2d6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:59:46.055952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16931716-db71-4fb5-b115-c2c969f6c2d6 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:59:46.079878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 707/1406] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:59:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 465 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 13:59:48.184626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f07f29-cf10-409b-a476-4391839d1aae None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:48.185181 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f07f29-cf10-409b-a476-4391839d1aae None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:48.185181 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82f07f29-cf10-409b-a476-4391839d1aae None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:48.192391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797d992-993b-4b59-a27f-201e1890ad5c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:48.192936 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797d992-993b-4b59-a27f-201e1890ad5c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:48.192936 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4797d992-993b-4b59-a27f-201e1890ad5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:48.196351 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 700/1407] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 13:59:48 2022] GET /identity => generated 270 bytes in 13 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 13:59:48.298099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4797d992-993b-4b59-a27f-201e1890ad5c None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 13:59:48.299856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4797d992-993b-4b59-a27f-201e1890ad5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:48.368751 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:48.431150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 708/1408] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 13:59:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 13:59:48.475441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f07f29-cf10-409b-a476-4391839d1aae None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 13:59:48.476982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-82f07f29-cf10-409b-a476-4391839d1aae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:48.642759 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82f07f29-cf10-409b-a476-4391839d1aae None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 13:59:48.720806 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:48.777369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb0998d-c04b-4019-a49a-3e1cbcc97e44 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 13:59:48.778123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0998d-c04b-4019-a49a-3e1cbcc97e44 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 13:59:48.778485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0998d-c04b-4019-a49a-3e1cbcc97e44 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 13:59:48.778821 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb0998d-c04b-4019-a49a-3e1cbcc97e44 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 13:59:48.780715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ffb0998d-c04b-4019-a49a-3e1cbcc97e44 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 13:59:48.964382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0998d-c04b-4019-a49a-3e1cbcc97e44 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 13:59:48.965924 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb0998d-c04b-4019-a49a-3e1cbcc97e44 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 13:59:48.985338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 701/1409] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 13:59:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 13:59:58.429241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=23 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 13:59:58.983234 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=64 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:05.170067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fedec-1ba1-4543-bec2-da5227c9cde8 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:05.170658 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fedec-1ba1-4543-bec2-da5227c9cde8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:05.170658 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634fedec-1ba1-4543-bec2-da5227c9cde8 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:05.172062 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 709/1410] 10.222.0.49 () {54 vars in 918 bytes} [Mon Jul 11 14:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:00:05.180930 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24a4c2-b47c-4c02-81d3-7bae4e646ea6 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:05.181426 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24a4c2-b47c-4c02-81d3-7bae4e646ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:05.181589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db24a4c2-b47c-4c02-81d3-7bae4e646ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:05.258659 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-db24a4c2-b47c-4c02-81d3-7bae4e646ea6 None None] MFA Rules not processed for user `ee29b7f1cea64d6781e1df5c8021363f`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:05.259891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-db24a4c2-b47c-4c02-81d3-7bae4e646ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:05.366019 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 702/1411] 10.222.0.49 () {58 vars in 1015 bytes} [Mon Jul 11 14:00:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:05.380354 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0af17b-f021-4eb5-bedc-fcf3cd2e14b6 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:05.380654 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0af17b-f021-4eb5-bedc-fcf3cd2e14b6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:05.380939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0af17b-f021-4eb5-bedc-fcf3cd2e14b6 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:05.385503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 710/1412] 10.222.0.49 () {54 vars in 918 bytes} [Mon Jul 11 14:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:00:05.387107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d0485-4e02-4112-9903-66a8516c00b1 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:05.387404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d0485-4e02-4112-9903-66a8516c00b1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:05.387683 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68d0485-4e02-4112-9903-66a8516c00b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:05.458780 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:05.461599 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b68d0485-4e02-4112-9903-66a8516c00b1 None None] MFA Rules not processed for user `ef5dac3f311e43809a46995bab56e780`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:05.463470 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b68d0485-4e02-4112-9903-66a8516c00b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:05.577180 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 703/1413] 10.222.0.49 () {58 vars in 1015 bytes} [Mon Jul 11 14:00:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:08.066660 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d5563-7b93-4f39-90cd-ba0950dbb4e4 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:08.067236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d5563-7b93-4f39-90cd-ba0950dbb4e4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:08.067236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea7d5563-7b93-4f39-90cd-ba0950dbb4e4 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:08.069158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 711/1414] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 14:00:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:00:08.075026 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dac3c2-ab21-4d9b-bc54-3b48da372afc None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:08.075508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dac3c2-ab21-4d9b-bc54-3b48da372afc None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:08.075609 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dac3c2-ab21-4d9b-bc54-3b48da372afc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:08.141210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e0dac3c2-ab21-4d9b-bc54-3b48da372afc None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:08.142449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e0dac3c2-ab21-4d9b-bc54-3b48da372afc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:08.198386 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:08.240679 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 704/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 14:00:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:08.282811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d5563-7b93-4f39-90cd-ba0950dbb4e4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:00:08.284387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ea7d5563-7b93-4f39-90cd-ba0950dbb4e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:08.434604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea7d5563-7b93-4f39-90cd-ba0950dbb4e4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:00:08.507881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:08.548901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb72b52c-fea7-4bfa-b881-31904f81f379 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:00:08.549558 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72b52c-fea7-4bfa-b881-31904f81f379 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:08.549847 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72b52c-fea7-4bfa-b881-31904f81f379 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:08.550136 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb72b52c-fea7-4bfa-b881-31904f81f379 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:08.551837 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bb72b52c-fea7-4bfa-b881-31904f81f379 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:08.760217 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb72b52c-fea7-4bfa-b881-31904f81f379 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:00:08.761659 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb72b52c-fea7-4bfa-b881-31904f81f379 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:00:08.779855 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 712/1416] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:00:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 498 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:00:09.667334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135e545-c76c-4f09-85f6-2e3eb066978c None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:09.667907 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135e545-c76c-4f09-85f6-2e3eb066978c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:09.667907 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d135e545-c76c-4f09-85f6-2e3eb066978c None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:09.669363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 705/1417] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 14:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:00:09.674173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c797e07-b403-420a-a0b0-1116d56529bf None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:09.674600 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c797e07-b403-420a-a0b0-1116d56529bf None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:09.674748 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c797e07-b403-420a-a0b0-1116d56529bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:09.747005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5c797e07-b403-420a-a0b0-1116d56529bf None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:09.748202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5c797e07-b403-420a-a0b0-1116d56529bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:09.775466 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:09.845778 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 713/1418] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 14:00:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:09.996535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135e545-c76c-4f09-85f6-2e3eb066978c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:00:09.998014 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d135e545-c76c-4f09-85f6-2e3eb066978c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:10.144199 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d135e545-c76c-4f09-85f6-2e3eb066978c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:00:10.183749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:10.257945 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d8ac78-0ab5-4164-bee8-a94651511ee2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:00:10.258547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8ac78-0ab5-4164-bee8-a94651511ee2 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:10.258833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8ac78-0ab5-4164-bee8-a94651511ee2 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:10.259108 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d8ac78-0ab5-4164-bee8-a94651511ee2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:10.260775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a0d8ac78-0ab5-4164-bee8-a94651511ee2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:10.348762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8ac78-0ab5-4164-bee8-a94651511ee2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:00:10.350578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d8ac78-0ab5-4164-bee8-a94651511ee2 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:00:10.368845 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 706/1419] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:00:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:00:11.248545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a109cc4-a280-4c66-a6f2-256f2ab5cd42 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:11.249113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a109cc4-a280-4c66-a6f2-256f2ab5cd42 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:11.249113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a109cc4-a280-4c66-a6f2-256f2ab5cd42 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:11.250571 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 714/1420] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 14:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:00:11.255432 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d958c-a4aa-4f94-893b-1b7897c3ae86 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:11.255832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d958c-a4aa-4f94-893b-1b7897c3ae86 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:11.255989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-132d958c-a4aa-4f94-893b-1b7897c3ae86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:11.308766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:11.330247 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-132d958c-a4aa-4f94-893b-1b7897c3ae86 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:11.331423 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-132d958c-a4aa-4f94-893b-1b7897c3ae86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:11.428137 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 707/1421] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 14:00:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:11.579517 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a109cc4-a280-4c66-a6f2-256f2ab5cd42 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:00:11.581198 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1a109cc4-a280-4c66-a6f2-256f2ab5cd42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:11.729496 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a109cc4-a280-4c66-a6f2-256f2ab5cd42 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:00:11.742145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:11.845350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e296087-72bb-49b4-a891-bb0dc9b4b717 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:00:11.845933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e296087-72bb-49b4-a891-bb0dc9b4b717 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:11.846199 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e296087-72bb-49b4-a891-bb0dc9b4b717 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:11.846508 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e296087-72bb-49b4-a891-bb0dc9b4b717 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:11.848158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6e296087-72bb-49b4-a891-bb0dc9b4b717 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:11.938702 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e296087-72bb-49b4-a891-bb0dc9b4b717 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:00:11.940300 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e296087-72bb-49b4-a891-bb0dc9b4b717 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:00:11.958684 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 715/1422] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:00:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:00:12.819461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995f46a-bdca-4350-81c2-534adf262fa9 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:12.820022 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995f46a-bdca-4350-81c2-534adf262fa9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:12.820022 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5995f46a-bdca-4350-81c2-534adf262fa9 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:12.821412 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 708/1423] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 14:00:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:00:12.826442 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc2e90-5a93-4bc8-8aac-4d0de504cfd4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:12.826864 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc2e90-5a93-4bc8-8aac-4d0de504cfd4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:12.827006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0fc2e90-5a93-4bc8-8aac-4d0de504cfd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:12.846315 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:12.908081 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c0fc2e90-5a93-4bc8-8aac-4d0de504cfd4 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:12.909298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c0fc2e90-5a93-4bc8-8aac-4d0de504cfd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:13.008029 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 716/1424] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 14:00:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 183 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:13.162764 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995f46a-bdca-4350-81c2-534adf262fa9 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:00:13.164327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5995f46a-bdca-4350-81c2-534adf262fa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:13.282412 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:13.324040 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5995f46a-bdca-4350-81c2-534adf262fa9 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:00:13.446757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812a0b7d-7df1-4a1f-82cf-0583f8bb5e54 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:00:13.447319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a0b7d-7df1-4a1f-82cf-0583f8bb5e54 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:13.447600 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a0b7d-7df1-4a1f-82cf-0583f8bb5e54 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:13.447871 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812a0b7d-7df1-4a1f-82cf-0583f8bb5e54 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:13.449537 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-812a0b7d-7df1-4a1f-82cf-0583f8bb5e54 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:13.539484 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:13.555912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a0b7d-7df1-4a1f-82cf-0583f8bb5e54 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:00:13.557565 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812a0b7d-7df1-4a1f-82cf-0583f8bb5e54 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:00:13.576061 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 709/1425] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:00:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:00:14.470550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb831a-0da2-40d7-a5ba-2ab150fc7689 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:14.471208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb831a-0da2-40d7-a5ba-2ab150fc7689 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:14.471208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfb831a-0da2-40d7-a5ba-2ab150fc7689 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:14.472499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 717/1426] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 14:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:00:14.478029 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b9b51-346b-4161-87d7-8b4a5d9b1f80 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:14.478535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b9b51-346b-4161-87d7-8b4a5d9b1f80 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:14.478662 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9b9b51-346b-4161-87d7-8b4a5d9b1f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:14.558792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5f9b9b51-346b-4161-87d7-8b4a5d9b1f80 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:14.560024 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5f9b9b51-346b-4161-87d7-8b4a5d9b1f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:14.663927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 710/1427] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 14:00:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:14.816697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb831a-0da2-40d7-a5ba-2ab150fc7689 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:00:14.818824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-acfb831a-0da2-40d7-a5ba-2ab150fc7689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:14.907851 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:14.973100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfb831a-0da2-40d7-a5ba-2ab150fc7689 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:00:15.097670 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1943117-1e7f-43f8-ac2c-0591a4b7fe60 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:00:15.098359 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1943117-1e7f-43f8-ac2c-0591a4b7fe60 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:15.098640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1943117-1e7f-43f8-ac2c-0591a4b7fe60 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:15.099403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1943117-1e7f-43f8-ac2c-0591a4b7fe60 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:15.101132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a1943117-1e7f-43f8-ac2c-0591a4b7fe60 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:15.150148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:15.193824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1943117-1e7f-43f8-ac2c-0591a4b7fe60 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:00:15.195461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1943117-1e7f-43f8-ac2c-0591a4b7fe60 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:00:15.214855 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 718/1428] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:00:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:00:16.019207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96288c9-6823-4859-a794-9f8af6036e96 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:16.019766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96288c9-6823-4859-a794-9f8af6036e96 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:16.019766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c96288c9-6823-4859-a794-9f8af6036e96 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:16.021718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 711/1429] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 14:00:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:00:16.026745 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ee770-410a-4ac3-bd3d-685cc4942a59 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:16.027144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ee770-410a-4ac3-bd3d-685cc4942a59 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:16.027368 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-847ee770-410a-4ac3-bd3d-685cc4942a59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:16.102686 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-847ee770-410a-4ac3-bd3d-685cc4942a59 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:16.103912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-847ee770-410a-4ac3-bd3d-685cc4942a59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:16.199996 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 719/1430] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 14:00:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:16.354290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96288c9-6823-4859-a794-9f8af6036e96 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:00:16.355839 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c96288c9-6823-4859-a794-9f8af6036e96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:16.406017 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:16.512091 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c96288c9-6823-4859-a794-9f8af6036e96 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:00:16.649901 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cef42f5f-d027-4967-96da-f82f9cc9a863 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:00:16.650590 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef42f5f-d027-4967-96da-f82f9cc9a863 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:16.650873 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef42f5f-d027-4967-96da-f82f9cc9a863 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:16.651152 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cef42f5f-d027-4967-96da-f82f9cc9a863 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:16.652846 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-cef42f5f-d027-4967-96da-f82f9cc9a863 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:16.659940 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:16.753081 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef42f5f-d027-4967-96da-f82f9cc9a863 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:00:16.754772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cef42f5f-d027-4967-96da-f82f9cc9a863 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:00:16.775166 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 712/1431] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:00:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:00:17.590559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1a34a-517d-47d8-afc1-0168ca3f04c3 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:17.591160 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1a34a-517d-47d8-afc1-0168ca3f04c3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:17.591160 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba1a34a-517d-47d8-afc1-0168ca3f04c3 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:17.592560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 720/1432] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 14:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:00:17.597474 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdafd48-3812-4e77-909f-9c76a9472bb2 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:17.597901 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdafd48-3812-4e77-909f-9c76a9472bb2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:17.598036 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecdafd48-3812-4e77-909f-9c76a9472bb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:17.672633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ecdafd48-3812-4e77-909f-9c76a9472bb2 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:17.673764 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ecdafd48-3812-4e77-909f-9c76a9472bb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:17.742362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:17.889340 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 713/1433] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 14:00:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 293 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:18.040079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1a34a-517d-47d8-afc1-0168ca3f04c3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:00:18.041519 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fba1a34a-517d-47d8-afc1-0168ca3f04c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:18.060706 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:18.192047 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba1a34a-517d-47d8-afc1-0168ca3f04c3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:00:18.281504 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:18.308852 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-901a71e4-8ee7-42cb-8ad4-012b50899b62 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:00:18.309430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a71e4-8ee7-42cb-8ad4-012b50899b62 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:18.309715 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a71e4-8ee7-42cb-8ad4-012b50899b62 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:18.309992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-901a71e4-8ee7-42cb-8ad4-012b50899b62 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:18.311669 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-901a71e4-8ee7-42cb-8ad4-012b50899b62 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:18.400583 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a71e4-8ee7-42cb-8ad4-012b50899b62 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:00:18.402158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-901a71e4-8ee7-42cb-8ad4-012b50899b62 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:00:18.419666 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 721/1434] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:00:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 380 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:00:19.227533 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e0d29-88a1-4895-9487-8c73b592f3ce None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:19.228099 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e0d29-88a1-4895-9487-8c73b592f3ce None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:19.228099 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-337e0d29-88a1-4895-9487-8c73b592f3ce None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:19.229478 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 714/1435] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 14:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:00:19.235064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9d280-912c-4b20-8d86-05586e4b6a33 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:19.235510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9d280-912c-4b20-8d86-05586e4b6a33 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:19.235625 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb9d280-912c-4b20-8d86-05586e4b6a33 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:19.314708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-efb9d280-912c-4b20-8d86-05586e4b6a33 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:19.315935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-efb9d280-912c-4b20-8d86-05586e4b6a33 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:19.359800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:19.415537 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 722/1436] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 14:00:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:19.570262 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e0d29-88a1-4895-9487-8c73b592f3ce None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:00:19.571843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-337e0d29-88a1-4895-9487-8c73b592f3ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:19.719546 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-337e0d29-88a1-4895-9487-8c73b592f3ce None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:00:19.773429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:19.834867 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0344ccb-75ae-4237-aa87-ac881ca0720d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:00:19.835521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0344ccb-75ae-4237-aa87-ac881ca0720d service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:19.835802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0344ccb-75ae-4237-aa87-ac881ca0720d service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:19.836078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0344ccb-75ae-4237-aa87-ac881ca0720d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:19.837720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e0344ccb-75ae-4237-aa87-ac881ca0720d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:19.925576 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0344ccb-75ae-4237-aa87-ac881ca0720d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:00:19.927744 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0344ccb-75ae-4237-aa87-ac881ca0720d service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:00:19.946176 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 715/1437] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:00:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 376 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:00:20.758312 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c6aad-047e-4863-a3c4-31d802391a4e None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:20.758967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c6aad-047e-4863-a3c4-31d802391a4e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:20.758967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15c6aad-047e-4863-a3c4-31d802391a4e None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:20.760318 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 723/1438] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 14:00:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:00:20.765398 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca40487-035c-4135-879d-1dcbb63ab779 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:20.765826 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca40487-035c-4135-879d-1dcbb63ab779 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:20.765960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca40487-035c-4135-879d-1dcbb63ab779 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:20.841332 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-2ca40487-035c-4135-879d-1dcbb63ab779 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:20.842556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2ca40487-035c-4135-879d-1dcbb63ab779 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:20.844551 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:20.939366 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 716/1439] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 14:00:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:21.092388 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c6aad-047e-4863-a3c4-31d802391a4e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:00:21.093917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e15c6aad-047e-4863-a3c4-31d802391a4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:21.240899 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15c6aad-047e-4863-a3c4-31d802391a4e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:00:21.268450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:21.357148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8cbbced-d63e-483b-b80c-224bb22b60be service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:00:21.357800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbbced-d63e-483b-b80c-224bb22b60be service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:21.358079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbbced-d63e-483b-b80c-224bb22b60be service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:21.358380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8cbbced-d63e-483b-b80c-224bb22b60be service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:21.360013 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f8cbbced-d63e-483b-b80c-224bb22b60be service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:21.449370 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cbbced-d63e-483b-b80c-224bb22b60be service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:00:21.451015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8cbbced-d63e-483b-b80c-224bb22b60be service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:00:21.470370 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 724/1440] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:00:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:00:23.411231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f509ec-114f-487a-9026-b4ae5947f426 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:23.411817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f509ec-114f-487a-9026-b4ae5947f426 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:23.411817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f509ec-114f-487a-9026-b4ae5947f426 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:23.413163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 717/1441] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 14:00:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:00:23.418740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cb2be-83e7-44af-b59d-813bbc0127ac None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:23.419170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cb2be-83e7-44af-b59d-813bbc0127ac None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:23.419310 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d43cb2be-83e7-44af-b59d-813bbc0127ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:23.460503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:23.485895 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d43cb2be-83e7-44af-b59d-813bbc0127ac None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:23.487146 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d43cb2be-83e7-44af-b59d-813bbc0127ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:23.583530 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 725/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 14:00:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:23.622050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f509ec-114f-487a-9026-b4ae5947f426 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:00:23.623648 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-63f509ec-114f-487a-9026-b4ae5947f426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:23.728736 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f509ec-114f-487a-9026-b4ae5947f426 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:00:23.759602 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:23.805115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee1bf9d3-5634-43bd-8351-7358edc90a65 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:00:23.805829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bf9d3-5634-43bd-8351-7358edc90a65 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:23.806175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bf9d3-5634-43bd-8351-7358edc90a65 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:23.806531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee1bf9d3-5634-43bd-8351-7358edc90a65 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:23.808417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ee1bf9d3-5634-43bd-8351-7358edc90a65 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:23.932602 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1bf9d3-5634-43bd-8351-7358edc90a65 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:00:23.934080 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee1bf9d3-5634-43bd-8351-7358edc90a65 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:00:23.954919 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 718/1443] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:00:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:00:26.134803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff290e1-418e-453f-aa3f-ec5b23e858fc None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:26.135327 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff290e1-418e-453f-aa3f-ec5b23e858fc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:26.135463 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff290e1-418e-453f-aa3f-ec5b23e858fc None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:26.136751 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 726/1444] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 14:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:00:26.142061 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4ac55-a65f-48f6-97d7-7421f36748bf None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:26.142542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4ac55-a65f-48f6-97d7-7421f36748bf None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:26.142641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b4ac55-a65f-48f6-97d7-7421f36748bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:26.210056 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c2b4ac55-a65f-48f6-97d7-7421f36748bf None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:26.211331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c2b4ac55-a65f-48f6-97d7-7421f36748bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:26.225076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:26.314584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 719/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 14:00:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:26.352191 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff290e1-418e-453f-aa3f-ec5b23e858fc None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:00:26.353724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2ff290e1-418e-453f-aa3f-ec5b23e858fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:26.459816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff290e1-418e-453f-aa3f-ec5b23e858fc None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:00:26.471870 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:26.535207 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a398e456-9887-4828-9f6b-5c0d1fd24bfc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:00:26.535765 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398e456-9887-4828-9f6b-5c0d1fd24bfc service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:26.536588 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398e456-9887-4828-9f6b-5c0d1fd24bfc service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:26.536949 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a398e456-9887-4828-9f6b-5c0d1fd24bfc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:26.538660 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a398e456-9887-4828-9f6b-5c0d1fd24bfc service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:26.658141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398e456-9887-4828-9f6b-5c0d1fd24bfc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:00:26.659618 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a398e456-9887-4828-9f6b-5c0d1fd24bfc service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:00:26.679302 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 727/1446] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:00:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:00:29.176432 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56e39b-ce2b-4b34-9288-6a731e15d857 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:29.177563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56e39b-ce2b-4b34-9288-6a731e15d857 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:29.177665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c56e39b-ce2b-4b34-9288-6a731e15d857 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:29.179302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 720/1447] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 14:00:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:00:29.184564 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787bb06-987b-4592-b25c-b81b7c932aed None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:29.184992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787bb06-987b-4592-b25c-b81b7c932aed None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:29.185135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d787bb06-987b-4592-b25c-b81b7c932aed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:29.231838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:29.250812 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d787bb06-987b-4592-b25c-b81b7c932aed None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:29.252039 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d787bb06-987b-4592-b25c-b81b7c932aed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:29.352275 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 728/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 14:00:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:29.390977 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56e39b-ce2b-4b34-9288-6a731e15d857 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:00:29.392546 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0c56e39b-ce2b-4b34-9288-6a731e15d857 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:29.497710 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c56e39b-ce2b-4b34-9288-6a731e15d857 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:00:29.537529 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:29.572005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56c00b89-d96a-4dac-861f-61724bd79993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:00:29.572657 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c00b89-d96a-4dac-861f-61724bd79993 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:29.572947 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c00b89-d96a-4dac-861f-61724bd79993 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:29.573224 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56c00b89-d96a-4dac-861f-61724bd79993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:29.574911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-56c00b89-d96a-4dac-861f-61724bd79993 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:29.685168 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c00b89-d96a-4dac-861f-61724bd79993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:00:29.686653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56c00b89-d96a-4dac-861f-61724bd79993 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:00:29.705778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 721/1449] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:00:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:00:31.889693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709c2f7-3b43-41ce-8876-e1279f100698 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:31.890241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709c2f7-3b43-41ce-8876-e1279f100698 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:31.890241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b709c2f7-3b43-41ce-8876-e1279f100698 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:31.891714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 729/1450] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 14:00:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:00:31.897290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db437b4-447d-4258-ae80-b06b0352f2c9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:31.897745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db437b4-447d-4258-ae80-b06b0352f2c9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:31.897846 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db437b4-447d-4258-ae80-b06b0352f2c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:31.961970 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8db437b4-447d-4258-ae80-b06b0352f2c9 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:31.963213 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8db437b4-447d-4258-ae80-b06b0352f2c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:31.983875 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:32.061676 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 722/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 14:00:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:32.100629 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709c2f7-3b43-41ce-8876-e1279f100698 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:00:32.102231 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b709c2f7-3b43-41ce-8876-e1279f100698 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:32.208923 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b709c2f7-3b43-41ce-8876-e1279f100698 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:00:32.230232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:32.284944 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2190a3cd-0ec3-44d7-b2c0-df084a1f0388 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:00:32.285620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190a3cd-0ec3-44d7-b2c0-df084a1f0388 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:32.285891 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190a3cd-0ec3-44d7-b2c0-df084a1f0388 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:32.286172 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2190a3cd-0ec3-44d7-b2c0-df084a1f0388 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:32.287887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2190a3cd-0ec3-44d7-b2c0-df084a1f0388 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:32.399224 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190a3cd-0ec3-44d7-b2c0-df084a1f0388 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:00:32.400680 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2190a3cd-0ec3-44d7-b2c0-df084a1f0388 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:00:32.419844 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 730/1452] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:00:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:00:34.625693 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293fa83-9e69-402e-ae23-2ce11085b9ef None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:34.626259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293fa83-9e69-402e-ae23-2ce11085b9ef None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:34.626259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4293fa83-9e69-402e-ae23-2ce11085b9ef None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:34.628207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 723/1453] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 14:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:00:34.633810 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7ca35-2988-4063-821a-1033ce266fff None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:34.634244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7ca35-2988-4063-821a-1033ce266fff None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:34.634404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff7ca35-2988-4063-821a-1033ce266fff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:34.697938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:34.701107 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6ff7ca35-2988-4063-821a-1033ce266fff None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:34.702704 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6ff7ca35-2988-4063-821a-1033ce266fff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:34.801156 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 731/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 14:00:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:34.840354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293fa83-9e69-402e-ae23-2ce11085b9ef None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:00:34.841903 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4293fa83-9e69-402e-ae23-2ce11085b9ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:34.961653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4293fa83-9e69-402e-ae23-2ce11085b9ef None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:00:35.010942 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:35.036180 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faac742c-5be3-40a9-b2d9-5fddb0101b3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:00:35.036836 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac742c-5be3-40a9-b2d9-5fddb0101b3a service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:35.037124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac742c-5be3-40a9-b2d9-5fddb0101b3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:35.037405 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faac742c-5be3-40a9-b2d9-5fddb0101b3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:35.039126 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-faac742c-5be3-40a9-b2d9-5fddb0101b3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:35.151494 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac742c-5be3-40a9-b2d9-5fddb0101b3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:00:35.152927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faac742c-5be3-40a9-b2d9-5fddb0101b3a service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:00:35.171886 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 724/1455] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:00:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:00:37.311707 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52dc73-8195-4f8d-8f40-3c9a58e837c9 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:37.312264 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52dc73-8195-4f8d-8f40-3c9a58e837c9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:37.312264 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f52dc73-8195-4f8d-8f40-3c9a58e837c9 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:37.313639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 732/1456] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 14:00:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:00:37.319032 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11ec2d-29ae-4def-8be0-79f8775765ac None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:37.319423 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11ec2d-29ae-4def-8be0-79f8775765ac None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:37.319577 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c11ec2d-29ae-4def-8be0-79f8775765ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:37.384293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5c11ec2d-29ae-4def-8be0-79f8775765ac None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:37.385419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5c11ec2d-29ae-4def-8be0-79f8775765ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:37.414783 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:37.484360 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 725/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 14:00:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:37.522133 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52dc73-8195-4f8d-8f40-3c9a58e837c9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:00:37.523793 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0f52dc73-8195-4f8d-8f40-3c9a58e837c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:37.628511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f52dc73-8195-4f8d-8f40-3c9a58e837c9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:00:37.657540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:37.702636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6afa776-9b3a-4c73-bcb1-d8322cf7bb3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:00:37.703278 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afa776-9b3a-4c73-bcb1-d8322cf7bb3a service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:37.703563 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afa776-9b3a-4c73-bcb1-d8322cf7bb3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:37.703842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6afa776-9b3a-4c73-bcb1-d8322cf7bb3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:37.705488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f6afa776-9b3a-4c73-bcb1-d8322cf7bb3a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:37.820157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afa776-9b3a-4c73-bcb1-d8322cf7bb3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:00:37.821599 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6afa776-9b3a-4c73-bcb1-d8322cf7bb3a service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:00:37.841096 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 733/1458] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:00:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:00:39.958053 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2699d94-9914-4ae5-8253-22a1d895e108 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:39.958776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2699d94-9914-4ae5-8253-22a1d895e108 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:39.958776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2699d94-9914-4ae5-8253-22a1d895e108 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:39.960282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 726/1459] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 14:00:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:00:39.966334 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7485ec-7fa4-4c79-8c0e-f0c731987828 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:39.966878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7485ec-7fa4-4c79-8c0e-f0c731987828 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:39.966878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7485ec-7fa4-4c79-8c0e-f0c731987828 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:40.041644 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0a7485ec-7fa4-4c79-8c0e-f0c731987828 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:40.042956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0a7485ec-7fa4-4c79-8c0e-f0c731987828 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:40.053018 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:40.146215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 734/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 14:00:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:40.184192 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2699d94-9914-4ae5-8253-22a1d895e108 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:00:40.185759 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c2699d94-9914-4ae5-8253-22a1d895e108 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:40.312201 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2699d94-9914-4ae5-8253-22a1d895e108 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:00:40.372756 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:40.394275 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93afefb3-704b-49c5-ba32-0b3744ef9dd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:00:40.395003 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afefb3-704b-49c5-ba32-0b3744ef9dd9 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:40.395285 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afefb3-704b-49c5-ba32-0b3744ef9dd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:40.395556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93afefb3-704b-49c5-ba32-0b3744ef9dd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:40.397414 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-93afefb3-704b-49c5-ba32-0b3744ef9dd9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:40.518629 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93afefb3-704b-49c5-ba32-0b3744ef9dd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:00:40.520140 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93afefb3-704b-49c5-ba32-0b3744ef9dd9 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:00:40.540848 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 727/1461] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:00:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 357 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:00:42.663241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a24717-7a7c-40b5-b5f7-cd49b4b04f3d None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:42.663797 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a24717-7a7c-40b5-b5f7-cd49b4b04f3d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:42.663797 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a24717-7a7c-40b5-b5f7-cd49b4b04f3d None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:42.665209 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 735/1462] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 14:00:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:00:42.671068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a3769-5953-44e6-a982-eca25e0249cb None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:42.671598 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a3769-5953-44e6-a982-eca25e0249cb None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:42.671706 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c00a3769-5953-44e6-a982-eca25e0249cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:42.739100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c00a3769-5953-44e6-a982-eca25e0249cb None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:42.740353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c00a3769-5953-44e6-a982-eca25e0249cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:42.781610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:42.843938 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 728/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 14:00:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:42.882749 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a24717-7a7c-40b5-b5f7-cd49b4b04f3d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:00:42.884338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-14a24717-7a7c-40b5-b5f7-cd49b4b04f3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:42.994079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a24717-7a7c-40b5-b5f7-cd49b4b04f3d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:00:43.036380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:43.082778 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d57bfcb-ca0d-4359-be89-14266667b629 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:00:43.083442 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57bfcb-ca0d-4359-be89-14266667b629 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:43.083729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57bfcb-ca0d-4359-be89-14266667b629 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:43.084007 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d57bfcb-ca0d-4359-be89-14266667b629 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:43.085745 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7d57bfcb-ca0d-4359-be89-14266667b629 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:43.213505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57bfcb-ca0d-4359-be89-14266667b629 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:00:43.215055 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d57bfcb-ca0d-4359-be89-14266667b629 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:00:43.237090 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 736/1464] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:00:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:00:45.373278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606423cb-d195-466e-9c31-84f6b2865bab None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:45.373808 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606423cb-d195-466e-9c31-84f6b2865bab None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:45.373808 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606423cb-d195-466e-9c31-84f6b2865bab None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:45.375218 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 729/1465] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 14:00:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 14:00:45.380520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe7bba-f85e-4356-9906-5f92a3505197 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:45.380933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe7bba-f85e-4356-9906-5f92a3505197 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:45.381144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fe7bba-f85e-4356-9906-5f92a3505197 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:45.445992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-76fe7bba-f85e-4356-9906-5f92a3505197 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:45.447239 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-76fe7bba-f85e-4356-9906-5f92a3505197 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:45.466123 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:45.548869 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 737/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 14:00:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:00:45.585325 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606423cb-d195-466e-9c31-84f6b2865bab None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:00:45.586910 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-606423cb-d195-466e-9c31-84f6b2865bab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:45.694542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606423cb-d195-466e-9c31-84f6b2865bab None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:00:45.761150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:45.769155 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9848da2e-d9da-4248-9a02-2da207c08237 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:00:45.770483 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848da2e-d9da-4248-9a02-2da207c08237 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:45.770769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848da2e-d9da-4248-9a02-2da207c08237 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:45.771047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9848da2e-d9da-4248-9a02-2da207c08237 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:45.772721 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9848da2e-d9da-4248-9a02-2da207c08237 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:45.883318 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848da2e-d9da-4248-9a02-2da207c08237 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:00:45.884771 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9848da2e-d9da-4248-9a02-2da207c08237 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:00:45.906174 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 730/1467] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:00:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:00:47.986513 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d812040-f1d9-456a-a2b6-bcc5f1dfed22 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:47.987074 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d812040-f1d9-456a-a2b6-bcc5f1dfed22 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:47.987074 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d812040-f1d9-456a-a2b6-bcc5f1dfed22 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:47.988485 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 738/1468] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 14:00:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 14:00:47.994057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869488b-0107-414c-a6e0-0d0d1456192d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:47.994562 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869488b-0107-414c-a6e0-0d0d1456192d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:47.994661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d869488b-0107-414c-a6e0-0d0d1456192d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:48.060267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-d869488b-0107-414c-a6e0-0d0d1456192d None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:00:48.061493 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d869488b-0107-414c-a6e0-0d0d1456192d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:48.111068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:48.160171 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 731/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 14:00:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:00:48.197644 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d812040-f1d9-456a-a2b6-bcc5f1dfed22 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:00:48.199220 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6d812040-f1d9-456a-a2b6-bcc5f1dfed22 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:48.310447 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d812040-f1d9-456a-a2b6-bcc5f1dfed22 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:00:48.363024 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:48.392343 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bc45d6-a6b1-4d19-9a83-c364bfb16e66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:00:48.393008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc45d6-a6b1-4d19-9a83-c364bfb16e66 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:00:48.393295 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc45d6-a6b1-4d19-9a83-c364bfb16e66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:00:48.393600 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bc45d6-a6b1-4d19-9a83-c364bfb16e66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:00:48.395363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f5bc45d6-a6b1-4d19-9a83-c364bfb16e66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:00:48.507598 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc45d6-a6b1-4d19-9a83-c364bfb16e66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:00:48.509077 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bc45d6-a6b1-4d19-9a83-c364bfb16e66 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:00:48.527343 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 739/1470] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:00:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 330 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:00:58.158295 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=22 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:00:58.524992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=66 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:01:08.405064 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc0f05-fc4e-4f50-bf8f-6a88b67221f5 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:08.405628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc0f05-fc4e-4f50-bf8f-6a88b67221f5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:08.405628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10cc0f05-fc4e-4f50-bf8f-6a88b67221f5 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:08.407076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 732/1471] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 14:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:01:08.412214 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b58e40-5a4c-4738-b891-1e4959f707fe None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:08.412628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b58e40-5a4c-4738-b891-1e4959f707fe None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:08.412752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7b58e40-5a4c-4738-b891-1e4959f707fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:08.478697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f7b58e40-5a4c-4738-b891-1e4959f707fe None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:01:08.479921 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f7b58e40-5a4c-4738-b891-1e4959f707fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:08.577222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 740/1472] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 14:01:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:01:08.660391 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbab62-1021-4f18-9993-bffd66197358 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:08.660685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbab62-1021-4f18-9993-bffd66197358 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:08.660970 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21dbab62-1021-4f18-9993-bffd66197358 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:08.662401 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 733/1473] 10.222.0.49 () {54 vars in 962 bytes} [Mon Jul 11 14:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:01:08.667155 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720a2e5-eea9-436f-9d71-e6df96f1743e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:08.667456 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720a2e5-eea9-436f-9d71-e6df96f1743e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:08.667743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9720a2e5-eea9-436f-9d71-e6df96f1743e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:08.742571 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9720a2e5-eea9-436f-9d71-e6df96f1743e None None] MFA Rules not processed for user `f80d59fec4994e689e857ba2087d3a23`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:01:08.743785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9720a2e5-eea9-436f-9d71-e6df96f1743e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:08.764954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:01:08.838499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 741/1474] 10.222.0.49 () {58 vars in 1059 bytes} [Mon Jul 11 14:01:08 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:01:08.846052 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbab62-1021-4f18-9993-bffd66197358 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:01:08.847659 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-21dbab62-1021-4f18-9993-bffd66197358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:08.956917 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21dbab62-1021-4f18-9993-bffd66197358 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:01:09.030757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2214ce72-26d6-4cb4-b3ba-10270c49c48b service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f80d59fec4994e689e857ba2087d3a23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:01:09.031438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214ce72-26d6-4cb4-b3ba-10270c49c48b service glance] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:09.031725 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214ce72-26d6-4cb4-b3ba-10270c49c48b service glance] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:09.032008 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2214ce72-26d6-4cb4-b3ba-10270c49c48b service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:09.033663 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2214ce72-26d6-4cb4-b3ba-10270c49c48b service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:09.088208 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:01:09.143583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214ce72-26d6-4cb4-b3ba-10270c49c48b service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:01:09.145070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2214ce72-26d6-4cb4-b3ba-10270c49c48b service glance] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:01:09.163959 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 734/1475] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:01:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 318 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:01:11.272835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163297a-9750-4d2f-90c7-c6ebf7473813 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:11.273404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163297a-9750-4d2f-90c7-c6ebf7473813 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:11.273404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4163297a-9750-4d2f-90c7-c6ebf7473813 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:11.274815 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 742/1476] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 14:01:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:01:11.280118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa49029-38c1-4f55-bee1-6c1368e52332 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:11.280565 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa49029-38c1-4f55-bee1-6c1368e52332 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:11.280710 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa49029-38c1-4f55-bee1-6c1368e52332 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:11.344073 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9fa49029-38c1-4f55-bee1-6c1368e52332 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:01:11.345196 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9fa49029-38c1-4f55-bee1-6c1368e52332 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:11.441834 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 735/1477] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 14:01:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:01:11.465828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163297a-9750-4d2f-90c7-c6ebf7473813 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:01:11.467399 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4163297a-9750-4d2f-90c7-c6ebf7473813 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:11.590982 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4163297a-9750-4d2f-90c7-c6ebf7473813 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:01:11.640341 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:01:11.688287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e44a579-434d-49ee-b10a-bf3a6e41a382 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:01:11.688954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44a579-434d-49ee-b10a-bf3a6e41a382 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:11.689241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44a579-434d-49ee-b10a-bf3a6e41a382 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:11.689517 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e44a579-434d-49ee-b10a-bf3a6e41a382 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:11.690260 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44a579-434d-49ee-b10a-bf3a6e41a382 admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:01:11.692636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-2e44a579-434d-49ee-b10a-bf3a6e41a382 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:01:11.692636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: warnings.warn(msg) Jul 11 14:01:11.692636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: Jul 11 14:01:11.693781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e44a579-434d-49ee-b10a-bf3a6e41a382 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:01:11.712903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 743/1478] 10.222.0.49 () {56 vars in 1108 bytes} [Mon Jul 11 14:01:11 2022] GET /identity/v3/projects => generated 2553 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:01:13.767485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00113a-4e16-4e8e-b583-c85517638a4c None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:13.768058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00113a-4e16-4e8e-b583-c85517638a4c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:13.768058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba00113a-4e16-4e8e-b583-c85517638a4c None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:13.769435 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 736/1479] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 14:01:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:01:13.774744 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf61a9-4f42-4bbb-872e-d205853dcd01 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:13.775263 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf61a9-4f42-4bbb-872e-d205853dcd01 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:13.775404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95bf61a9-4f42-4bbb-872e-d205853dcd01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:13.841290 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-95bf61a9-4f42-4bbb-872e-d205853dcd01 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:01:13.842515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-95bf61a9-4f42-4bbb-872e-d205853dcd01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:13.943260 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 744/1480] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 14:01:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:01:13.987260 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00113a-4e16-4e8e-b583-c85517638a4c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:01:13.988841 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ba00113a-4e16-4e8e-b583-c85517638a4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:13.999242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:01:14.135004 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba00113a-4e16-4e8e-b583-c85517638a4c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:01:14.215116 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:01:14.247723 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aded0df-7679-4dc2-9b26-4b8756be87e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:01:14.248374 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aded0df-7679-4dc2-9b26-4b8756be87e9 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:14.248658 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aded0df-7679-4dc2-9b26-4b8756be87e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:14.248933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aded0df-7679-4dc2-9b26-4b8756be87e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:14.250615 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1aded0df-7679-4dc2-9b26-4b8756be87e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:14.340017 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aded0df-7679-4dc2-9b26-4b8756be87e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:01:14.341533 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aded0df-7679-4dc2-9b26-4b8756be87e9 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:01:14.359190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 737/1481] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:01:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:01:16.582082 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559e124-23b5-4692-b3d8-1cb34608c88d None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:16.582776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559e124-23b5-4692-b3d8-1cb34608c88d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:16.582776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e559e124-23b5-4692-b3d8-1cb34608c88d None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:16.584068 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 745/1482] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 14:01:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:01:16.589402 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61aeb3-b154-47c4-af24-ace292349448 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:16.589866 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61aeb3-b154-47c4-af24-ace292349448 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:16.589981 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec61aeb3-b154-47c4-af24-ace292349448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:16.657466 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ec61aeb3-b154-47c4-af24-ace292349448 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:01:16.658760 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ec61aeb3-b154-47c4-af24-ace292349448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:16.705929 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:01:16.756636 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 738/1483] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 14:01:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:01:16.793249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559e124-23b5-4692-b3d8-1cb34608c88d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:01:16.794824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e559e124-23b5-4692-b3d8-1cb34608c88d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:16.820652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:01:16.906986 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e559e124-23b5-4692-b3d8-1cb34608c88d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:01:16.981555 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d43d137-5aba-4931-8887-f0172d001a17 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:01:16.982204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43d137-5aba-4931-8887-f0172d001a17 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:16.982516 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43d137-5aba-4931-8887-f0172d001a17 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:16.982796 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d43d137-5aba-4931-8887-f0172d001a17 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:16.984446 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6d43d137-5aba-4931-8887-f0172d001a17 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:17.057856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:01:17.093269 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43d137-5aba-4931-8887-f0172d001a17 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:01:17.094756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d43d137-5aba-4931-8887-f0172d001a17 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:01:17.115193 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 746/1484] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:01:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:01:19.223274 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8eefd-6515-42dc-a6a7-1a37f8759528 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:19.223778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8eefd-6515-42dc-a6a7-1a37f8759528 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:19.223897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f8eefd-6515-42dc-a6a7-1a37f8759528 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:19.225251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 739/1485] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 14:01:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:01:19.230560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ba004-10e5-433f-8062-83a30a6edc52 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:19.231092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ba004-10e5-433f-8062-83a30a6edc52 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:19.231092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc9ba004-10e5-433f-8062-83a30a6edc52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:19.296983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-fc9ba004-10e5-433f-8062-83a30a6edc52 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:01:19.298196 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fc9ba004-10e5-433f-8062-83a30a6edc52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:19.395332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 747/1486] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 14:01:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:01:19.433739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8eefd-6515-42dc-a6a7-1a37f8759528 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:01:19.435338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-28f8eefd-6515-42dc-a6a7-1a37f8759528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:19.538881 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f8eefd-6515-42dc-a6a7-1a37f8759528 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:01:19.611375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:01:19.613447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace7ac95-ecf8-4ca2-bae1-f6c74c9e354a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:01:19.614117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7ac95-ecf8-4ca2-bae1-f6c74c9e354a service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:19.614656 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7ac95-ecf8-4ca2-bae1-f6c74c9e354a service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:19.615139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace7ac95-ecf8-4ca2-bae1-f6c74c9e354a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:19.617043 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ace7ac95-ecf8-4ca2-bae1-f6c74c9e354a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:19.727658 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7ac95-ecf8-4ca2-bae1-f6c74c9e354a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:01:19.729019 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace7ac95-ecf8-4ca2-bae1-f6c74c9e354a service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:01:19.747496 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 740/1487] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:01:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:01:21.927209 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30311423-8179-4f43-8de6-7cb86514d834 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:21.927825 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30311423-8179-4f43-8de6-7cb86514d834 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:21.927825 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30311423-8179-4f43-8de6-7cb86514d834 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:21.929448 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 748/1488] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 14:01:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:01:21.935042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4dfb8-44c8-42fd-9d0c-d637c28e023e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:21.935503 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4dfb8-44c8-42fd-9d0c-d637c28e023e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:21.935625 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad4dfb8-44c8-42fd-9d0c-d637c28e023e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:22.000533 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3ad4dfb8-44c8-42fd-9d0c-d637c28e023e None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:01:22.001774 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3ad4dfb8-44c8-42fd-9d0c-d637c28e023e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:22.058076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:01:22.102294 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 741/1489] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 14:01:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:01:22.142207 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30311423-8179-4f43-8de6-7cb86514d834 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:01:22.143791 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-30311423-8179-4f43-8de6-7cb86514d834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:22.180904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:01:22.248444 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30311423-8179-4f43-8de6-7cb86514d834 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:01:22.322488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01b07263-1e18-4312-ba73-513c346920e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:01:22.323138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b07263-1e18-4312-ba73-513c346920e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:22.323421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b07263-1e18-4312-ba73-513c346920e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:22.323693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01b07263-1e18-4312-ba73-513c346920e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:22.325348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-01b07263-1e18-4312-ba73-513c346920e6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:22.405572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:01:22.436535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b07263-1e18-4312-ba73-513c346920e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:01:22.437926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01b07263-1e18-4312-ba73-513c346920e6 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:01:22.456744 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 749/1490] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:01:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:01:24.699422 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225877b8-84b9-45b2-82e6-22141ad4682a None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:24.699904 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225877b8-84b9-45b2-82e6-22141ad4682a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:24.700033 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225877b8-84b9-45b2-82e6-22141ad4682a None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:24.701353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 742/1491] 10.222.0.49 () {54 vars in 926 bytes} [Mon Jul 11 14:01:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:01:24.707239 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b6765-2358-4bd8-b392-12fde8d8769f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:24.707695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b6765-2358-4bd8-b392-12fde8d8769f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:24.707806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9b6765-2358-4bd8-b392-12fde8d8769f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:24.773397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4b9b6765-2358-4bd8-b392-12fde8d8769f None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:01:24.774643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4b9b6765-2358-4bd8-b392-12fde8d8769f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:24.874190 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 750/1492] 10.222.0.49 () {58 vars in 1023 bytes} [Mon Jul 11 14:01:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:01:24.913140 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225877b8-84b9-45b2-82e6-22141ad4682a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:01:24.914726 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-225877b8-84b9-45b2-82e6-22141ad4682a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:25.017993 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225877b8-84b9-45b2-82e6-22141ad4682a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:01:25.094237 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317d9be0-f940-4f23-b952-b587a5e14d7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:01:25.094928 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d9be0-f940-4f23-b952-b587a5e14d7d service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:01:25.095212 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d9be0-f940-4f23-b952-b587a5e14d7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:01:25.095487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317d9be0-f940-4f23-b952-b587a5e14d7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:01:25.097229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-317d9be0-f940-4f23-b952-b587a5e14d7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:01:25.106441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:01:25.208741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d9be0-f940-4f23-b952-b587a5e14d7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:01:25.210145 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317d9be0-f940-4f23-b952-b587a5e14d7d service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:01:25.229691 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 743/1493] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:01:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:01:34.872327 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=82 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:01:35.227321 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=50 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:02:59.132501 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff36a87-0ed9-4451-989a-dada7882c572 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:02:59.133069 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff36a87-0ed9-4451-989a-dada7882c572 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:02:59.133069 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff36a87-0ed9-4451-989a-dada7882c572 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:02:59.209288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7ff36a87-0ed9-4451-989a-dada7882c572 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:02:59.210543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7ff36a87-0ed9-4451-989a-dada7882c572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:02:59.311038 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 751/1494] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:02:59.319226 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54720bcc-ded1-495a-b746-b5a992982bd9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:02:59.319767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54720bcc-ded1-495a-b746-b5a992982bd9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:02:59.319767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54720bcc-ded1-495a-b746-b5a992982bd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:02:59.394664 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-54720bcc-ded1-495a-b746-b5a992982bd9 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:02:59.395890 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-54720bcc-ded1-495a-b746-b5a992982bd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:02:59.492678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 744/1495] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:02:59.497306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff36a87-0ed9-4451-989a-dada7882c572 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:02:59.498559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7ff36a87-0ed9-4451-989a-dada7882c572 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:02:59.509957 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff36a87-0ed9-4451-989a-dada7882c572 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:02:59.572491 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:02:59.608887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f97330-0ae4-4d12-a5a7-65024c9f43c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:02:59.609559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f97330-0ae4-4d12-a5a7-65024c9f43c3 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:02:59.609846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f97330-0ae4-4d12-a5a7-65024c9f43c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:02:59.610140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f97330-0ae4-4d12-a5a7-65024c9f43c3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:02:59.610899 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f97330-0ae4-4d12-a5a7-65024c9f43c3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:02:59.611211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f97330-0ae4-4d12-a5a7-65024c9f43c3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:02:59.612236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-36f97330-0ae4-4d12-a5a7-65024c9f43c3 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:02:59.612236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: warnings.warn(msg) Jul 11 14:02:59.612236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: Jul 11 14:02:59.613137 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f97330-0ae4-4d12-a5a7-65024c9f43c3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:02:59.624946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 752/1496] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:02:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 128 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:02:59.633207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54720bcc-ded1-495a-b746-b5a992982bd9 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:02:59.642993 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54720bcc-ded1-495a-b746-b5a992982bd9 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:02:59.711695 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:02:59.746478 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d024e1-bf66-40aa-b9f7-1e942ed3f807 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:02:59.747141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d024e1-bf66-40aa-b9f7-1e942ed3f807 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:02:59.747423 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d024e1-bf66-40aa-b9f7-1e942ed3f807 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:02:59.747695 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d024e1-bf66-40aa-b9f7-1e942ed3f807 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:02:59.748374 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d024e1-bf66-40aa-b9f7-1e942ed3f807 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:02:59.750121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: WARNING py.warnings [None req-65d024e1-bf66-40aa-b9f7-1e942ed3f807 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:02:59.750121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: warnings.warn(msg) Jul 11 14:02:59.750121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: Jul 11 14:02:59.751343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d024e1-bf66-40aa-b9f7-1e942ed3f807 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:02:59.805881 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 745/1497] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:02:59 2022] POST /identity/v3/projects => generated 377 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:02:59.810070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f97330-0ae4-4d12-a5a7-65024c9f43c3 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:02:59.811637 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-36f97330-0ae4-4d12-a5a7-65024c9f43c3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:02:59.845928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f97330-0ae4-4d12-a5a7-65024c9f43c3 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:02:59.949665 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc56571-1a33-4b12-9b5b-956bf36b16ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:02:59.950417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc56571-1a33-4b12-9b5b-956bf36b16ce admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:02:59.950704 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc56571-1a33-4b12-9b5b-956bf36b16ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:02:59.950979 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc56571-1a33-4b12-9b5b-956bf36b16ce admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:02:59.951693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc56571-1a33-4b12-9b5b-956bf36b16ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:02:59.953344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-fcc56571-1a33-4b12-9b5b-956bf36b16ce admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:02:59.953344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: warnings.warn(msg) Jul 11 14:02:59.953344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: Jul 11 14:02:59.954868 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc56571-1a33-4b12-9b5b-956bf36b16ce admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:02:59.997090 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 753/1498] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:02:59 2022] POST /identity/v3/users => generated 289 bytes in 187 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:03:00.001770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d024e1-bf66-40aa-b9f7-1e942ed3f807 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:03:00.003377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-65d024e1-bf66-40aa-b9f7-1e942ed3f807 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:03:00.068615 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d024e1-bf66-40aa-b9f7-1e942ed3f807 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:03:00.157452 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:03:00.169528 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae7872-2a0b-4a97-897c-d5b2a83647ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:03:00.170200 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae7872-2a0b-4a97-897c-d5b2a83647ef admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:03:00.170519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae7872-2a0b-4a97-897c-d5b2a83647ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:03:00.170791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1ae7872-2a0b-4a97-897c-d5b2a83647ef admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:03:00.171554 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae7872-2a0b-4a97-897c-d5b2a83647ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:03:00.172596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: WARNING py.warnings [None req-c1ae7872-2a0b-4a97-897c-d5b2a83647ef admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:03:00.172596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: warnings.warn(msg) Jul 11 14:03:00.172596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: Jul 11 14:03:00.173474 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1ae7872-2a0b-4a97-897c-d5b2a83647ef admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:03:00.186577 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 746/1499] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:03:00.195105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc56571-1a33-4b12-9b5b-956bf36b16ce admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:03:00.196650 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fcc56571-1a33-4b12-9b5b-956bf36b16ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:03:00.209108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc56571-1a33-4b12-9b5b-956bf36b16ce admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:03:00.234686 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:03:00.306607 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-872797bb-22cc-42ac-bfe5-8e2dd0039d1d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:03:00.307202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872797bb-22cc-42ac-bfe5-8e2dd0039d1d admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:03:00.307486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872797bb-22cc-42ac-bfe5-8e2dd0039d1d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:03:00.307763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-872797bb-22cc-42ac-bfe5-8e2dd0039d1d admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:03:00.308496 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872797bb-22cc-42ac-bfe5-8e2dd0039d1d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:03:00.309509 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-872797bb-22cc-42ac-bfe5-8e2dd0039d1d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:03:00.309509 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: warnings.warn(msg) Jul 11 14:03:00.309509 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: Jul 11 14:03:00.310428 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-872797bb-22cc-42ac-bfe5-8e2dd0039d1d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:03:00.323402 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 754/1500] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:03:00 2022] GET /identity/v3/roles => generated 1420 bytes in 129 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:03:00.327587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae7872-2a0b-4a97-897c-d5b2a83647ef admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:03:00.329118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c1ae7872-2a0b-4a97-897c-d5b2a83647ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:03:00.362236 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1ae7872-2a0b-4a97-897c-d5b2a83647ef admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:03:00.455913 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c75bde-d3b5-4119-8ef5-62a1f6c0f945 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:03:00.456799 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c75bde-d3b5-4119-8ef5-62a1f6c0f945 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:03:00.457083 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c75bde-d3b5-4119-8ef5-62a1f6c0f945 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:03:00.457361 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c75bde-d3b5-4119-8ef5-62a1f6c0f945 admin admin] PATH_INFO: `/v3/projects/6fc2ed6b3cc04972985f29f6ad586f63/users/15b95b5fbbae4d9cb0c9f9125a8475c8/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:03:00.479447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c75bde-d3b5-4119-8ef5-62a1f6c0f945 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6fc2ed6b3cc04972985f29f6ad586f63, user_id=15b95b5fbbae4d9cb0c9f9125a8475c8, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:03:00.480487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: WARNING py.warnings [None req-93c75bde-d3b5-4119-8ef5-62a1f6c0f945 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:03:00.480487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: warnings.warn(msg) Jul 11 14:03:00.480487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: Jul 11 14:03:00.482438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c75bde-d3b5-4119-8ef5-62a1f6c0f945 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:03:00.507209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 747/1501] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:03:00 2022] PUT /identity/v3/projects/6fc2ed6b3cc04972985f29f6ad586f63/users/15b95b5fbbae4d9cb0c9f9125a8475c8/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:03:00.519267 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ecd59-31da-4ab4-98d0-59fa0ed80ee8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:03:00.519568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ecd59-31da-4ab4-98d0-59fa0ed80ee8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:03:00.519852 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff2ecd59-31da-4ab4-98d0-59fa0ed80ee8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:03:00.575418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ff2ecd59-31da-4ab4-98d0-59fa0ed80ee8 None None] MFA Rules not processed for user `15b95b5fbbae4d9cb0c9f9125a8475c8`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:03:00.576554 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ff2ecd59-31da-4ab4-98d0-59fa0ed80ee8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:03:00.659102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 755/1502] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:03:00 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:03:00.672716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233bbc1-f6c9-4608-82e2-7da9feaddd61 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:03:00.673004 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233bbc1-f6c9-4608-82e2-7da9feaddd61 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:03:00.673280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9233bbc1-f6c9-4608-82e2-7da9feaddd61 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:03:00.674711 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 748/1503] 10.222.0.49 () {54 vars in 963 bytes} [Mon Jul 11 14:03:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:03:00.679768 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf07dcb-2e2c-4485-915e-b14e1190f140 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:03:00.680065 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf07dcb-2e2c-4485-915e-b14e1190f140 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:03:00.680348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf07dcb-2e2c-4485-915e-b14e1190f140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:03:00.710099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:03:00.758847 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-fcf07dcb-2e2c-4485-915e-b14e1190f140 None None] MFA Rules not processed for user `6bb1616f632d4c67828cce3fc41945b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:03:00.760085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fcf07dcb-2e2c-4485-915e-b14e1190f140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:03:00.841872 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 756/1504] 10.222.0.49 () {58 vars in 1060 bytes} [Mon Jul 11 14:03:00 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:03:00.850166 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233bbc1-f6c9-4608-82e2-7da9feaddd61 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:03:00.851793 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9233bbc1-f6c9-4608-82e2-7da9feaddd61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:03:00.911742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:03:00.968783 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9233bbc1-f6c9-4608-82e2-7da9feaddd61 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:03:01.053805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef77cf-7950-452d-a293-937a7ac85a82 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:03:01.054499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef77cf-7950-452d-a293-937a7ac85a82 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:03:01.054786 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef77cf-7950-452d-a293-937a7ac85a82 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:03:01.055061 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37ef77cf-7950-452d-a293-937a7ac85a82 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:03:01.056753 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-37ef77cf-7950-452d-a293-937a7ac85a82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:03:01.162680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef77cf-7950-452d-a293-937a7ac85a82 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:03:01.164133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37ef77cf-7950-452d-a293-937a7ac85a82 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:03:01.167944 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:03:01.184123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 749/1505] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:03:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:03:01.229282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bbe31-4f9e-4212-81d7-f7f63dab664c None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:03:01.229786 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bbe31-4f9e-4212-81d7-f7f63dab664c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:03:01.229909 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268bbe31-4f9e-4212-81d7-f7f63dab664c None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:03:01.231146 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 757/1506] 10.222.0.49 () {52 vars in 829 bytes} [Mon Jul 11 14:03:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:03:01.234985 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef77cf-7950-452d-a293-937a7ac85a82 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:03:01.236195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-37ef77cf-7950-452d-a293-937a7ac85a82 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:03:01.335024 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37ef77cf-7950-452d-a293-937a7ac85a82 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:03:01.426965 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf55a2-907d-497d-848b-dcff7a6686a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:03:01.427753 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf55a2-907d-497d-848b-dcff7a6686a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:03:01.428288 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf55a2-907d-497d-848b-dcff7a6686a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:03:01.428558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cdf55a2-907d-497d-848b-dcff7a6686a6 admin admin] PATH_INFO: `/v3/users/15b95b5fbbae4d9cb0c9f9125a8475c8` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:03:01.441267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf55a2-907d-497d-848b-dcff7a6686a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15b95b5fbbae4d9cb0c9f9125a8475c8)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:03:01.442290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: WARNING py.warnings [None req-7cdf55a2-907d-497d-848b-dcff7a6686a6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:03:01.442290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: warnings.warn(msg) Jul 11 14:03:01.442290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: Jul 11 14:03:01.443527 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cdf55a2-907d-497d-848b-dcff7a6686a6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:03:01.453741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:03:01.506492 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7cdf55a2-907d-497d-848b-dcff7a6686a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15b95b5fbbae4d9cb0c9f9125a8475c8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:03:01.524631 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7cdf55a2-907d-497d-848b-dcff7a6686a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15b95b5fbbae4d9cb0c9f9125a8475c8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:03:01.533621 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7cdf55a2-907d-497d-848b-dcff7a6686a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15b95b5fbbae4d9cb0c9f9125a8475c8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:03:01.534134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7cdf55a2-907d-497d-848b-dcff7a6686a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15b95b5fbbae4d9cb0c9f9125a8475c8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:03:01.546351 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7cdf55a2-907d-497d-848b-dcff7a6686a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15b95b5fbbae4d9cb0c9f9125a8475c8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:03:01.551893 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 750/1507] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:03:01 2022] DELETE /identity/v3/users/15b95b5fbbae4d9cb0c9f9125a8475c8 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:03:01.556553 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321aaf33-7cfb-43d7-a045-91302332ac27 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:03:01.556851 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321aaf33-7cfb-43d7-a045-91302332ac27 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:03:01.557138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-321aaf33-7cfb-43d7-a045-91302332ac27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:03:01.613014 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-321aaf33-7cfb-43d7-a045-91302332ac27 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:03:01.614238 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-321aaf33-7cfb-43d7-a045-91302332ac27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:03:01.693025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:03:01.706960 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 758/1508] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:03:01.716387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf55a2-907d-497d-848b-dcff7a6686a6 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:03:01.717643 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7cdf55a2-907d-497d-848b-dcff7a6686a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:03:01.818120 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cdf55a2-907d-497d-848b-dcff7a6686a6 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:03:01.875610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:03:01.899111 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fadfac-0208-4249-823c-f1f8bb5fe208 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:03:01.899784 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fadfac-0208-4249-823c-f1f8bb5fe208 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:03:01.900053 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fadfac-0208-4249-823c-f1f8bb5fe208 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:03:01.900332 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fadfac-0208-4249-823c-f1f8bb5fe208 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:03:01.902032 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-43fadfac-0208-4249-823c-f1f8bb5fe208 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:03:02.016035 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fadfac-0208-4249-823c-f1f8bb5fe208 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:03:02.017438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fadfac-0208-4249-823c-f1f8bb5fe208 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:03:02.036588 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 751/1509] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:03:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 321 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:03:02.399733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321aaf33-7cfb-43d7-a045-91302332ac27 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:03:02.401043 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-321aaf33-7cfb-43d7-a045-91302332ac27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:03:02.413165 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-321aaf33-7cfb-43d7-a045-91302332ac27 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:03:02.521277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630917ca-87d3-4ba5-adab-5264536dea21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:03:02.522147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630917ca-87d3-4ba5-adab-5264536dea21 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:03:02.522467 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630917ca-87d3-4ba5-adab-5264536dea21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:03:02.522752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630917ca-87d3-4ba5-adab-5264536dea21 admin admin] PATH_INFO: `/v3/projects/6fc2ed6b3cc04972985f29f6ad586f63` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:03:02.533867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630917ca-87d3-4ba5-adab-5264536dea21 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6fc2ed6b3cc04972985f29f6ad586f63)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:03:02.534884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-630917ca-87d3-4ba5-adab-5264536dea21 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:03:02.534884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: warnings.warn(msg) Jul 11 14:03:02.534884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: Jul 11 14:03:02.536098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630917ca-87d3-4ba5-adab-5264536dea21 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:03:02.609120 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-630917ca-87d3-4ba5-adab-5264536dea21 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6fc2ed6b3cc04972985f29f6ad586f63', 'request_id': 'req-630917ca-87d3-4ba5-adab-5264536dea21'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:03:02.615659 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-630917ca-87d3-4ba5-adab-5264536dea21 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6fc2ed6b3cc04972985f29f6ad586f63', 'request_id': 'req-630917ca-87d3-4ba5-adab-5264536dea21'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:03:02.625745 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-630917ca-87d3-4ba5-adab-5264536dea21 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6fc2ed6b3cc04972985f29f6ad586f63', 'request_id': 'req-630917ca-87d3-4ba5-adab-5264536dea21'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:03:02.659215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-630917ca-87d3-4ba5-adab-5264536dea21 admin admin] The token cache is being invalidate because project 6fc2ed6b3cc04972985f29f6ad586f63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:03:02.659507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-630917ca-87d3-4ba5-adab-5264536dea21 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:03:02.660043 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-630917ca-87d3-4ba5-adab-5264536dea21 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:03:02.661440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 759/1510] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:03:02 2022] DELETE /identity/v3/projects/6fc2ed6b3cc04972985f29f6ad586f63 => generated 0 bytes in 262 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:03:12.034179 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=62 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:03:12.657113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:04:47.267366 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e864a4-0e28-40c0-93b6-1169e38da412 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:04:47.267979 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e864a4-0e28-40c0-93b6-1169e38da412 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:04:47.268087 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e864a4-0e28-40c0-93b6-1169e38da412 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:04:47.269426 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 752/1511] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 14:04:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:04:47.274417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8691573-8fa6-4d7d-9193-3b0aa8fea39e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:04:47.274876 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8691573-8fa6-4d7d-9193-3b0aa8fea39e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:04:47.274982 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8691573-8fa6-4d7d-9193-3b0aa8fea39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:04:47.356147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a8691573-8fa6-4d7d-9193-3b0aa8fea39e None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:04:47.357287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a8691573-8fa6-4d7d-9193-3b0aa8fea39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:04:47.472207 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 760/1512] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 14:04:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:04:47.628673 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e864a4-0e28-40c0-93b6-1169e38da412 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:04:47.630887 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-69e864a4-0e28-40c0-93b6-1169e38da412 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:04:47.751396 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e864a4-0e28-40c0-93b6-1169e38da412 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:04:47.756969 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2cc2124-9d98-4e67-a6b8-ad809e1bdb9c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:04:47.757625 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc2124-9d98-4e67-a6b8-ad809e1bdb9c service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:04:47.757914 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc2124-9d98-4e67-a6b8-ad809e1bdb9c service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:04:47.758194 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2cc2124-9d98-4e67-a6b8-ad809e1bdb9c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:04:47.767567 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc2124-9d98-4e67-a6b8-ad809e1bdb9c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:04:47.769163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2cc2124-9d98-4e67-a6b8-ad809e1bdb9c service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:04:47.779495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 753/1513] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:04:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:04:48.624721 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bba3da-9a22-420e-90c9-434703c5d9a5 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:04:48.625286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bba3da-9a22-420e-90c9-434703c5d9a5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:04:48.625286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1bba3da-9a22-420e-90c9-434703c5d9a5 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:04:48.626693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 761/1514] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 14:04:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:04:48.631458 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a354c8a-a69c-44f7-b629-e27066a8fce6 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:04:48.631903 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a354c8a-a69c-44f7-b629-e27066a8fce6 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:04:48.632056 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a354c8a-a69c-44f7-b629-e27066a8fce6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:04:48.650446 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-6a354c8a-a69c-44f7-b629-e27066a8fce6 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:04:48.651657 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6a354c8a-a69c-44f7-b629-e27066a8fce6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:04:48.659561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 754/1515] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 14:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:04:48.807895 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bba3da-9a22-420e-90c9-434703c5d9a5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:04:48.822718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1bba3da-9a22-420e-90c9-434703c5d9a5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:04:48.828774 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-426cce2a-fd97-433e-9954-ca12155d2f08 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:04:48.829410 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cce2a-fd97-433e-9954-ca12155d2f08 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:04:48.829699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cce2a-fd97-433e-9954-ca12155d2f08 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:04:48.829981 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426cce2a-fd97-433e-9954-ca12155d2f08 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:04:48.838125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cce2a-fd97-433e-9954-ca12155d2f08 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:04:48.839746 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-426cce2a-fd97-433e-9954-ca12155d2f08 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:04:48.849162 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 762/1516] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:04:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:04:49.198915 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd34de5-c059-4b84-8e4c-0e726cd123c3 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:04:49.199209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd34de5-c059-4b84-8e4c-0e726cd123c3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:04:49.199494 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd34de5-c059-4b84-8e4c-0e726cd123c3 None None] PATH_INFO: `/v3` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:04:49.200828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 755/1517] 10.222.0.55 () {54 vars in 925 bytes} [Mon Jul 11 14:04:49 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:04:49.207074 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba50577-c938-43f2-b143-16883a2f7e5b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:04:49.207440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba50577-c938-43f2-b143-16883a2f7e5b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:04:49.207736 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba50577-c938-43f2-b143-16883a2f7e5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:04:49.265930 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4ba50577-c938-43f2-b143-16883a2f7e5b None None] MFA Rules not processed for user `ee29b7f1cea64d6781e1df5c8021363f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:04:49.267179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4ba50577-c938-43f2-b143-16883a2f7e5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:04:49.283718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:04:49.340021 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 763/1518] 10.222.0.55 () {58 vars in 1015 bytes} [Mon Jul 11 14:04:49 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:04:58.300248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b83d04-e54f-472b-8247-f9e19dcf710d None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:04:58.300816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b83d04-e54f-472b-8247-f9e19dcf710d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:04:58.300816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b83d04-e54f-472b-8247-f9e19dcf710d None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:04:58.302139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 756/1519] 10.222.0.49 () {54 vars in 960 bytes} [Mon Jul 11 14:04:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:04:58.307757 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1654bc9-1a0c-467e-9c56-3cca97a17eea None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:04:58.308212 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1654bc9-1a0c-467e-9c56-3cca97a17eea None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:04:58.308319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1654bc9-1a0c-467e-9c56-3cca97a17eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:04:58.326854 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c1654bc9-1a0c-467e-9c56-3cca97a17eea None None] MFA Rules not processed for user `ee29b7f1cea64d6781e1df5c8021363f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:04:58.328083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c1654bc9-1a0c-467e-9c56-3cca97a17eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:04:58.336805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 764/1520] 10.222.0.49 () {58 vars in 1057 bytes} [Mon Jul 11 14:04:58 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:04:58.344859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b83d04-e54f-472b-8247-f9e19dcf710d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:04:58.358336 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b83d04-e54f-472b-8247-f9e19dcf710d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:04:58.363804 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20b15560-975f-4f3a-8bfd-036bfa1ae6dd service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee29b7f1cea64d6781e1df5c8021363f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:04:58.364439 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b15560-975f-4f3a-8bfd-036bfa1ae6dd service placement] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:04:58.364786 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b15560-975f-4f3a-8bfd-036bfa1ae6dd service placement] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:04:58.364995 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20b15560-975f-4f3a-8bfd-036bfa1ae6dd service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:04:58.373547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b15560-975f-4f3a-8bfd-036bfa1ae6dd service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:04:58.375220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20b15560-975f-4f3a-8bfd-036bfa1ae6dd service placement] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:04:58.385044 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 757/1521] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:04:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:05:00.084869 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e9130-aa2f-4c36-9518-edc67acbdd40 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:05:00.085434 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e9130-aa2f-4c36-9518-edc67acbdd40 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:05:00.085434 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93e9130-aa2f-4c36-9518-edc67acbdd40 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:05:00.086874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 765/1522] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 14:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:05:00.091989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68b933-7a0d-4773-adce-9da7d5414473 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:05:00.092491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68b933-7a0d-4773-adce-9da7d5414473 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:05:00.092603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f68b933-7a0d-4773-adce-9da7d5414473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:05:00.110907 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9f68b933-7a0d-4773-adce-9da7d5414473 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:05:00.112127 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9f68b933-7a0d-4773-adce-9da7d5414473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:05:00.120187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 758/1523] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 14:05:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:05:00.276811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e9130-aa2f-4c36-9518-edc67acbdd40 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:05:00.292542 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93e9130-aa2f-4c36-9518-edc67acbdd40 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:05:00.298358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07484b35-d246-43a4-8461-f1a13bd899e8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:05:00.299008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07484b35-d246-43a4-8461-f1a13bd899e8 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:05:00.299294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07484b35-d246-43a4-8461-f1a13bd899e8 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:05:00.299573 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07484b35-d246-43a4-8461-f1a13bd899e8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:05:00.308557 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07484b35-d246-43a4-8461-f1a13bd899e8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:05:00.310152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07484b35-d246-43a4-8461-f1a13bd899e8 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:05:00.319916 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 766/1524] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:05:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:05:01.158425 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6982002-0855-465c-9827-3ba6539d0ece None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:05:01.158985 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6982002-0855-465c-9827-3ba6539d0ece None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:05:01.158985 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6982002-0855-465c-9827-3ba6539d0ece None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:05:01.160387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 759/1525] 127.0.0.1 () {54 vars in 901 bytes} [Mon Jul 11 14:05:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:05:01.165247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83d075-bf6d-4101-9b00-e9379abc3473 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:05:01.165689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83d075-bf6d-4101-9b00-e9379abc3473 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:05:01.165809 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff83d075-bf6d-4101-9b00-e9379abc3473 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:05:01.184093 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ff83d075-bf6d-4101-9b00-e9379abc3473 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:05:01.185319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ff83d075-bf6d-4101-9b00-e9379abc3473 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:05:01.193516 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 767/1526] 127.0.0.1 () {58 vars in 998 bytes} [Mon Jul 11 14:05:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:05:01.346699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6982002-0855-465c-9827-3ba6539d0ece None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:05:01.360948 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6982002-0855-465c-9827-3ba6539d0ece None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:05:01.366417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66876d3b-b5c3-4e2b-8b33-fc794b4c0d9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:05:01.367068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66876d3b-b5c3-4e2b-8b33-fc794b4c0d9b service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:05:01.367382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66876d3b-b5c3-4e2b-8b33-fc794b4c0d9b service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:05:01.367660 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66876d3b-b5c3-4e2b-8b33-fc794b4c0d9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:05:01.377592 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66876d3b-b5c3-4e2b-8b33-fc794b4c0d9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:05:01.379261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66876d3b-b5c3-4e2b-8b33-fc794b4c0d9b service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:05:01.390566 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 760/1527] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:05:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:05:05.443126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44f906-32db-4efa-93a8-650919135d57 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:05:05.443690 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44f906-32db-4efa-93a8-650919135d57 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:05:05.443690 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b44f906-32db-4efa-93a8-650919135d57 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:05:05.445077 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 768/1528] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 14:05:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 14:05:05.450599 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e53c6-e775-4147-8c54-26e0c6427445 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:05:05.451062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e53c6-e775-4147-8c54-26e0c6427445 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:05:05.451169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e53c6-e775-4147-8c54-26e0c6427445 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:05:05.470937 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4d7e53c6-e775-4147-8c54-26e0c6427445 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:05:05.472163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4d7e53c6-e775-4147-8c54-26e0c6427445 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:05:05.480224 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 761/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 14:05:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:05:05.559631 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44f906-32db-4efa-93a8-650919135d57 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:05:05.562109 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6b44f906-32db-4efa-93a8-650919135d57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:05:05.650216 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b44f906-32db-4efa-93a8-650919135d57 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:05:05.654152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9e4df3-d9ae-4c7f-a8ff-05dbb8187a85 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:05:05.654894 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e4df3-d9ae-4c7f-a8ff-05dbb8187a85 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:05:05.655186 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e4df3-d9ae-4c7f-a8ff-05dbb8187a85 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:05:05.655466 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9e4df3-d9ae-4c7f-a8ff-05dbb8187a85 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:05:05.666176 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e4df3-d9ae-4c7f-a8ff-05dbb8187a85 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:05:05.667678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9e4df3-d9ae-4c7f-a8ff-05dbb8187a85 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:05:05.679303 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 769/1530] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:05:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:05:07.732427 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bb0ff-483a-411c-b3ac-c7944c83dc3d None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:05:07.733034 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bb0ff-483a-411c-b3ac-c7944c83dc3d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:05:07.733034 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb2bb0ff-483a-411c-b3ac-c7944c83dc3d None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:05:07.734632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 762/1531] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 14:05:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:05:07.740254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2adda-beca-4922-a961-041e46721a5a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:05:07.740730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2adda-beca-4922-a961-041e46721a5a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:05:07.740842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afd2adda-beca-4922-a961-041e46721a5a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:05:07.759024 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-afd2adda-beca-4922-a961-041e46721a5a None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:05:07.760232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-afd2adda-beca-4922-a961-041e46721a5a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:05:07.768722 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 770/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 14:05:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:05:07.805423 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bb0ff-483a-411c-b3ac-c7944c83dc3d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:05:07.817017 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb2bb0ff-483a-411c-b3ac-c7944c83dc3d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:05:07.820655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ef4d5e4-7be2-4b9f-9e75-4f04cb482a92 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:05:07.821296 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4d5e4-7be2-4b9f-9e75-4f04cb482a92 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:05:07.821580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4d5e4-7be2-4b9f-9e75-4f04cb482a92 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:05:07.821855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ef4d5e4-7be2-4b9f-9e75-4f04cb482a92 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:05:07.831314 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4d5e4-7be2-4b9f-9e75-4f04cb482a92 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:05:07.832757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ef4d5e4-7be2-4b9f-9e75-4f04cb482a92 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:05:07.842907 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 763/1533] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:05:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:05:11.621552 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857354c-34e9-4af4-8780-013ea2bca9c4 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:05:11.622126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857354c-34e9-4af4-8780-013ea2bca9c4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:05:11.622126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5857354c-34e9-4af4-8780-013ea2bca9c4 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:05:11.623546 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 771/1534] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 14:05:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:05:11.631486 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba9ee9-c7f3-4ee9-96d8-8f72e0442115 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:05:11.632001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba9ee9-c7f3-4ee9-96d8-8f72e0442115 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:05:11.632330 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6ba9ee9-c7f3-4ee9-96d8-8f72e0442115 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:05:11.658166 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b6ba9ee9-c7f3-4ee9-96d8-8f72e0442115 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:05:11.659986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b6ba9ee9-c7f3-4ee9-96d8-8f72e0442115 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:05:11.672471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 764/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 14:05:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:05:11.996965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857354c-34e9-4af4-8780-013ea2bca9c4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:05:11.999290 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5857354c-34e9-4af4-8780-013ea2bca9c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:05:12.075530 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:05:12.094864 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5857354c-34e9-4af4-8780-013ea2bca9c4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:05:12.099130 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc04e412-04a0-4369-b3f9-a01239137e31 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:05:12.099801 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04e412-04a0-4369-b3f9-a01239137e31 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:05:12.100089 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04e412-04a0-4369-b3f9-a01239137e31 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:05:12.100370 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc04e412-04a0-4369-b3f9-a01239137e31 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:05:12.110811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04e412-04a0-4369-b3f9-a01239137e31 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:05:12.112264 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc04e412-04a0-4369-b3f9-a01239137e31 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:05:12.122700 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 772/1536] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:05:11 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:05:14.238991 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de9714-5516-49e3-b511-f0b8451ea67d None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:05:14.239564 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de9714-5516-49e3-b511-f0b8451ea67d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:05:14.239564 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14de9714-5516-49e3-b511-f0b8451ea67d None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:05:14.240934 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 765/1537] 127.0.0.1 () {54 vars in 916 bytes} [Mon Jul 11 14:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 11 14:05:14.246640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5a0bb-bf0d-419f-b07f-ee9457181655 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:05:14.247150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5a0bb-bf0d-419f-b07f-ee9457181655 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:05:14.247264 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eff5a0bb-bf0d-419f-b07f-ee9457181655 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:05:14.266607 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-eff5a0bb-bf0d-419f-b07f-ee9457181655 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:05:14.267879 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-eff5a0bb-bf0d-419f-b07f-ee9457181655 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:05:14.276389 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 773/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Mon Jul 11 14:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:05:14.591629 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de9714-5516-49e3-b511-f0b8451ea67d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:05:14.593863 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-14de9714-5516-49e3-b511-f0b8451ea67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:05:14.601607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=84 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:05:14.608594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14de9714-5516-49e3-b511-f0b8451ea67d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:05:14.612298 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e54c8f-af0e-442d-bd24-7f7e08dd7a35 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:05:14.612924 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e54c8f-af0e-442d-bd24-7f7e08dd7a35 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:05:14.613215 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e54c8f-af0e-442d-bd24-7f7e08dd7a35 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:05:14.613499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e54c8f-af0e-442d-bd24-7f7e08dd7a35 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:05:14.624411 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e54c8f-af0e-442d-bd24-7f7e08dd7a35 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:05:14.625871 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e54c8f-af0e-442d-bd24-7f7e08dd7a35 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:05:14.635861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 766/1539] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:05:14 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:05:24.260589 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=11 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:05:24.631550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:45.440652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985c112-1747-49aa-af4e-1b316a452282 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:45.441319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985c112-1747-49aa-af4e-1b316a452282 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:45.441319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e985c112-1747-49aa-af4e-1b316a452282 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:45.470020 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e985c112-1747-49aa-af4e-1b316a452282 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:45.471699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e985c112-1747-49aa-af4e-1b316a452282 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:45.484240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 774/1540] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:45.493260 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bff441-9adf-43d2-8f79-196fc37ac925 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:45.493858 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bff441-9adf-43d2-8f79-196fc37ac925 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:45.493858 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27bff441-9adf-43d2-8f79-196fc37ac925 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:45.522097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-27bff441-9adf-43d2-8f79-196fc37ac925 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:45.523346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-27bff441-9adf-43d2-8f79-196fc37ac925 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:45.537617 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 767/1541] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:45.543278 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985c112-1747-49aa-af4e-1b316a452282 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:45.554574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e985c112-1747-49aa-af4e-1b316a452282 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:45.562430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba768b31-2382-42fa-b71c-bb56e361a1e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:45.563171 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba768b31-2382-42fa-b71c-bb56e361a1e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:45.563463 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba768b31-2382-42fa-b71c-bb56e361a1e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:45.563746 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba768b31-2382-42fa-b71c-bb56e361a1e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:45.565203 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba768b31-2382-42fa-b71c-bb56e361a1e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:06:45.565521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba768b31-2382-42fa-b71c-bb56e361a1e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:45.567324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba768b31-2382-42fa-b71c-bb56e361a1e9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:45.584310 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 775/1542] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:06:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:06:45.592748 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bff441-9adf-43d2-8f79-196fc37ac925 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:45.607941 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27bff441-9adf-43d2-8f79-196fc37ac925 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:45.619061 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c4e6fe-4be8-4af7-8352-8f0d086ef8ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:45.619756 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4e6fe-4be8-4af7-8352-8f0d086ef8ba admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:45.620046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4e6fe-4be8-4af7-8352-8f0d086ef8ba admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:45.620329 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c4e6fe-4be8-4af7-8352-8f0d086ef8ba admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:45.621061 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4e6fe-4be8-4af7-8352-8f0d086ef8ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:45.623045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c4e6fe-4be8-4af7-8352-8f0d086ef8ba admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:45.662579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26be90-eccc-4cd4-826c-b30317cc0728 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:45.662913 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26be90-eccc-4cd4-826c-b30317cc0728 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:45.663116 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f26be90-eccc-4cd4-826c-b30317cc0728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:45.679841 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 768/1543] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:06:45 2022] POST /identity/v3/projects => generated 385 bytes in 87 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:45.682186 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d2098-fee8-4e40-a435-59ec21976808 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:45.682521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d2098-fee8-4e40-a435-59ec21976808 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:45.682808 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7d2098-fee8-4e40-a435-59ec21976808 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:45.701857 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8f26be90-eccc-4cd4-826c-b30317cc0728 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:45.703136 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8f26be90-eccc-4cd4-826c-b30317cc0728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:45.713152 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ce7d2098-fee8-4e40-a435-59ec21976808 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:45.714419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ce7d2098-fee8-4e40-a435-59ec21976808 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:45.824091 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 776/1544] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:45.827328 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26be90-eccc-4cd4-826c-b30317cc0728 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:45.828823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 769/1545] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:45.836617 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6e06a-688e-44b4-9b33-d95868db2ded None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:45.839558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6e06a-688e-44b4-9b33-d95868db2ded None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:45.839854 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc6e06a-688e-44b4-9b33-d95868db2ded None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:45.849251 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f26be90-eccc-4cd4-826c-b30317cc0728 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:45.854564 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed49beac-2292-4882-85d2-bd13d52de213 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:45.855510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49beac-2292-4882-85d2-bd13d52de213 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:45.855795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49beac-2292-4882-85d2-bd13d52de213 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:45.856089 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed49beac-2292-4882-85d2-bd13d52de213 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:45.856891 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49beac-2292-4882-85d2-bd13d52de213 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:45.858852 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed49beac-2292-4882-85d2-bd13d52de213 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:45.862765 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9fc6e06a-688e-44b4-9b33-d95868db2ded None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:45.863976 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9fc6e06a-688e-44b4-9b33-d95868db2ded None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:45.879707 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 770/1546] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:45.884835 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6feb777-86d8-4727-bb5b-8da66872ef62 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:45.885114 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6feb777-86d8-4727-bb5b-8da66872ef62 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:45.885407 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6feb777-86d8-4727-bb5b-8da66872ef62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:45.896499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 777/1547] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:06:45 2022] POST /identity/v3/users => generated 293 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:45.898535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09130d-6c3e-4eb8-bfb3-8ceffd8d6281 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:45.899198 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09130d-6c3e-4eb8-bfb3-8ceffd8d6281 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:45.899729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed09130d-6c3e-4eb8-bfb3-8ceffd8d6281 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:45.905803 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a6feb777-86d8-4727-bb5b-8da66872ef62 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:45.907363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a6feb777-86d8-4727-bb5b-8da66872ef62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:45.918153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 771/1548] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:45.919525 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ed09130d-6c3e-4eb8-bfb3-8ceffd8d6281 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:45.920210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdc302-925d-485d-81b2-4fa4c31be67d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:45.920739 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ed09130d-6c3e-4eb8-bfb3-8ceffd8d6281 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:45.920902 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdc302-925d-485d-81b2-4fa4c31be67d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:45.921090 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cdc302-925d-485d-81b2-4fa4c31be67d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:45.930393 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 778/1549] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:45.931671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb545cff-3387-4580-a5d5-b33424071305 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:45.931958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb545cff-3387-4580-a5d5-b33424071305 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:45.932233 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb545cff-3387-4580-a5d5-b33424071305 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:45.941338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-07cdc302-925d-485d-81b2-4fa4c31be67d None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:45.942587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-07cdc302-925d-485d-81b2-4fa4c31be67d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:45.950876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 772/1550] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:45.951164 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-eb545cff-3387-4580-a5d5-b33424071305 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:45.952126 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc4186-75f6-4b06-aa6b-9470752543ea None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:45.952338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-eb545cff-3387-4580-a5d5-b33424071305 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:45.952456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc4186-75f6-4b06-aa6b-9470752543ea None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:45.952676 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24cc4186-75f6-4b06-aa6b-9470752543ea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:45.961743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 779/1551] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:45.963022 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70116c-2b9e-402d-83c3-4c299e565c0c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:45.963310 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70116c-2b9e-402d-83c3-4c299e565c0c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:45.963822 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d70116c-2b9e-402d-83c3-4c299e565c0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:45.971561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-24cc4186-75f6-4b06-aa6b-9470752543ea None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:45.972745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-24cc4186-75f6-4b06-aa6b-9470752543ea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:45.981288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9d70116c-2b9e-402d-83c3-4c299e565c0c None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:45.981677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 773/1552] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:45.982513 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9d70116c-2b9e-402d-83c3-4c299e565c0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:45.982788 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc4186-75f6-4b06-aa6b-9470752543ea None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:45.991248 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 780/1553] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:45.992490 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bdbe0-62bb-4f3d-be8a-e86cc60f2431 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:45.993324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bdbe0-62bb-4f3d-be8a-e86cc60f2431 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:45.993324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9bdbe0-62bb-4f3d-be8a-e86cc60f2431 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:45.993523 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24cc4186-75f6-4b06-aa6b-9470752543ea None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:45.998407 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f33c8946-7e13-4138-a6dd-abb052a03d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:45.999083 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c8946-7e13-4138-a6dd-abb052a03d2f admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:45.999371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c8946-7e13-4138-a6dd-abb052a03d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:45.999654 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f33c8946-7e13-4138-a6dd-abb052a03d2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.000412 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c8946-7e13-4138-a6dd-abb052a03d2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.002662 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f33c8946-7e13-4138-a6dd-abb052a03d2f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.012337 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3e9bdbe0-62bb-4f3d-be8a-e86cc60f2431 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:46.016248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 774/1554] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:06:46.016409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3e9bdbe0-62bb-4f3d-be8a-e86cc60f2431 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:46.019592 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c112e0-5c6f-4330-b120-df63212aa10a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.019951 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c112e0-5c6f-4330-b120-df63212aa10a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.020147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7c112e0-5c6f-4330-b120-df63212aa10a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.024753 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 781/1555] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:46.025510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bdbe0-62bb-4f3d-be8a-e86cc60f2431 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.037349 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9bdbe0-62bb-4f3d-be8a-e86cc60f2431 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.042051 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51de0a5-dc79-4a7c-a3b1-ea695da7bab7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.042742 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51de0a5-dc79-4a7c-a3b1-ea695da7bab7 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.042893 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b7c112e0-5c6f-4330-b120-df63212aa10a None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:46.043030 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51de0a5-dc79-4a7c-a3b1-ea695da7bab7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.043306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51de0a5-dc79-4a7c-a3b1-ea695da7bab7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.043981 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b7c112e0-5c6f-4330-b120-df63212aa10a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:46.044104 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51de0a5-dc79-4a7c-a3b1-ea695da7bab7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:06:46.044335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51de0a5-dc79-4a7c-a3b1-ea695da7bab7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.045942 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51de0a5-dc79-4a7c-a3b1-ea695da7bab7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.057346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 775/1556] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:46.058793 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 782/1557] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:06:46.059368 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74475772-a05a-497b-936b-b6322fcd41b5 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.059903 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74475772-a05a-497b-936b-b6322fcd41b5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.060237 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd919c8d-48e4-452f-8145-3c8e6af0d870 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.060526 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd919c8d-48e4-452f-8145-3c8e6af0d870 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.061090 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd919c8d-48e4-452f-8145-3c8e6af0d870 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.061240 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74475772-a05a-497b-936b-b6322fcd41b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.081111 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-74475772-a05a-497b-936b-b6322fcd41b5 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:46.082433 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-74475772-a05a-497b-936b-b6322fcd41b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:46.086482 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-dd919c8d-48e4-452f-8145-3c8e6af0d870 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:46.087664 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dd919c8d-48e4-452f-8145-3c8e6af0d870 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:46.090555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 776/1558] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:46.091443 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74475772-a05a-497b-936b-b6322fcd41b5 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.096795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 783/1559] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:46.097587 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd919c8d-48e4-452f-8145-3c8e6af0d870 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.104045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74475772-a05a-497b-936b-b6322fcd41b5 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.106683 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd919c8d-48e4-452f-8145-3c8e6af0d870 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.108792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc015b-f59c-4e3e-b103-d9ec08a9fa16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.109446 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc015b-f59c-4e3e-b103-d9ec08a9fa16 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.109735 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc015b-f59c-4e3e-b103-d9ec08a9fa16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.110014 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bddc015b-f59c-4e3e-b103-d9ec08a9fa16 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.110814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc015b-f59c-4e3e-b103-d9ec08a9fa16 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:06:46.111624 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc015b-f59c-4e3e-b103-d9ec08a9fa16 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.111748 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6113e-3213-4629-b69b-63cd127a5fc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.112329 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6113e-3213-4629-b69b-63cd127a5fc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.112597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6113e-3213-4629-b69b-63cd127a5fc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.112717 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: WARNING py.warnings [None req-bddc015b-f59c-4e3e-b103-d9ec08a9fa16 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:06:46.112717 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: warnings.warn(msg) Jul 11 14:06:46.112717 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: Jul 11 14:06:46.112971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6113e-3213-4629-b69b-63cd127a5fc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.113522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bddc015b-f59c-4e3e-b103-d9ec08a9fa16 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.113642 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6113e-3213-4629-b69b-63cd127a5fc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:06:46.113878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6113e-3213-4629-b69b-63cd127a5fc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.115977 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6113e-3213-4629-b69b-63cd127a5fc8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.125958 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 777/1560] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:06:46.127005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc015b-f59c-4e3e-b103-d9ec08a9fa16 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.128083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 784/1561] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:06:46.129384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac97442-25a2-4e73-93a9-5a14dfb1b9f2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.129675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac97442-25a2-4e73-93a9-5a14dfb1b9f2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.129959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac97442-25a2-4e73-93a9-5a14dfb1b9f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.138588 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bddc015b-f59c-4e3e-b103-d9ec08a9fa16 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.144027 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f85dc-a8fa-4dd7-86cd-9e063374bd2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.144730 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f85dc-a8fa-4dd7-86cd-9e063374bd2f admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.144989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f85dc-a8fa-4dd7-86cd-9e063374bd2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.145273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-459f85dc-a8fa-4dd7-86cd-9e063374bd2f admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.146014 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f85dc-a8fa-4dd7-86cd-9e063374bd2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.147685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-459f85dc-a8fa-4dd7-86cd-9e063374bd2f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.153252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3ac97442-25a2-4e73-93a9-5a14dfb1b9f2 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:46.154564 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3ac97442-25a2-4e73-93a9-5a14dfb1b9f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:46.159507 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 778/1562] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:06:46.160437 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f85dc-a8fa-4dd7-86cd-9e063374bd2f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.163601 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 785/1563] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:46.164431 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac97442-25a2-4e73-93a9-5a14dfb1b9f2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.172201 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-459f85dc-a8fa-4dd7-86cd-9e063374bd2f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.173901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac97442-25a2-4e73-93a9-5a14dfb1b9f2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.177544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd0900-c704-4038-9fad-f0fe53fa6600 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.178189 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd0900-c704-4038-9fad-f0fe53fa6600 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.178501 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd0900-c704-4038-9fad-f0fe53fa6600 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.178777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd0900-c704-4038-9fad-f0fe53fa6600 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.179488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953de572-f8cb-4f57-b396-8c81abee37c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.179623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd0900-c704-4038-9fad-f0fe53fa6600 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:06:46.179791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd0900-c704-4038-9fad-f0fe53fa6600 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.180152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953de572-f8cb-4f57-b396-8c81abee37c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.180440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953de572-f8cb-4f57-b396-8c81abee37c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.180719 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-953de572-f8cb-4f57-b396-8c81abee37c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.181375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd0900-c704-4038-9fad-f0fe53fa6600 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.181483 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953de572-f8cb-4f57-b396-8c81abee37c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.182468 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-953de572-f8cb-4f57-b396-8c81abee37c7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:06:46.182468 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: warnings.warn(msg) Jul 11 14:06:46.182468 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: Jul 11 14:06:46.184122 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-953de572-f8cb-4f57-b396-8c81abee37c7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.193775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 779/1564] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:06:46.194637 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd0900-c704-4038-9fad-f0fe53fa6600 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.206228 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd0900-c704-4038-9fad-f0fe53fa6600 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.210924 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec4b51-be09-4ed4-a52d-e80d1b622c7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.214591 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 786/1565] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:06:46 2022] POST /identity/v3/projects => generated 377 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:46.214745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec4b51-be09-4ed4-a52d-e80d1b622c7f admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.214745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec4b51-be09-4ed4-a52d-e80d1b622c7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.214745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bec4b51-be09-4ed4-a52d-e80d1b622c7f admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.214968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953de572-f8cb-4f57-b396-8c81abee37c7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.215050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec4b51-be09-4ed4-a52d-e80d1b622c7f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:06:46.215050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec4b51-be09-4ed4-a52d-e80d1b622c7f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.215050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bec4b51-be09-4ed4-a52d-e80d1b622c7f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.228447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 780/1566] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:06:46.229151 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec4b51-be09-4ed4-a52d-e80d1b622c7f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.239558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bec4b51-be09-4ed4-a52d-e80d1b622c7f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.257336 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:46.258980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:46.283089 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-953de572-f8cb-4f57-b396-8c81abee37c7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.287718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c259c-60fb-4bea-87f2-5781122b50d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.288387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c259c-60fb-4bea-87f2-5781122b50d8 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.288675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c259c-60fb-4bea-87f2-5781122b50d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.288954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a22c259c-60fb-4bea-87f2-5781122b50d8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.289689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c259c-60fb-4bea-87f2-5781122b50d8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:06:46.289998 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c259c-60fb-4bea-87f2-5781122b50d8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.291630 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a22c259c-60fb-4bea-87f2-5781122b50d8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.297835 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56cd2f-96ae-4d10-972a-0c8941c327aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.298549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56cd2f-96ae-4d10-972a-0c8941c327aa admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.298829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56cd2f-96ae-4d10-972a-0c8941c327aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.299101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d56cd2f-96ae-4d10-972a-0c8941c327aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.299786 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56cd2f-96ae-4d10-972a-0c8941c327aa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.301672 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d56cd2f-96ae-4d10-972a-0c8941c327aa admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.303421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 787/1567] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:06:46.304210 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c259c-60fb-4bea-87f2-5781122b50d8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.316267 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a22c259c-60fb-4bea-87f2-5781122b50d8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.321310 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa9483-db70-440e-889b-293c79546788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.321970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa9483-db70-440e-889b-293c79546788 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.322253 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa9483-db70-440e-889b-293c79546788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.322559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9aa9483-db70-440e-889b-293c79546788 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.323255 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa9483-db70-440e-889b-293c79546788 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.325291 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9aa9483-db70-440e-889b-293c79546788 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.332449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 781/1568] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:06:46 2022] POST /identity/v3/projects => generated 389 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:46.333332 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56cd2f-96ae-4d10-972a-0c8941c327aa admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.355324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 788/1569] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:06:46 2022] POST /identity/v3/projects => generated 377 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:46.355324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa9483-db70-440e-889b-293c79546788 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.395426 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d56cd2f-96ae-4d10-972a-0c8941c327aa admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.416479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9aa9483-db70-440e-889b-293c79546788 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.421135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9ecf2-6f46-4af8-b816-c030c270a605 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.421798 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9ecf2-6f46-4af8-b816-c030c270a605 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.422083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9ecf2-6f46-4af8-b816-c030c270a605 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.422448 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d9ecf2-6f46-4af8-b816-c030c270a605 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.423129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9ecf2-6f46-4af8-b816-c030c270a605 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:06:46.423500 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9ecf2-6f46-4af8-b816-c030c270a605 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.425113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d9ecf2-6f46-4af8-b816-c030c270a605 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.436963 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 789/1570] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:06:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:06:46.437772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9ecf2-6f46-4af8-b816-c030c270a605 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.448433 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d9ecf2-6f46-4af8-b816-c030c270a605 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.453147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464b614-17a1-48f2-8f3f-1133fec3193c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.453797 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464b614-17a1-48f2-8f3f-1133fec3193c admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.454082 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464b614-17a1-48f2-8f3f-1133fec3193c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.454400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7464b614-17a1-48f2-8f3f-1133fec3193c admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.455221 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464b614-17a1-48f2-8f3f-1133fec3193c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.457145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7464b614-17a1-48f2-8f3f-1133fec3193c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.460970 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082e4db-d513-4905-8eb6-deb2cc6d6272 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.462006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082e4db-d513-4905-8eb6-deb2cc6d6272 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.462293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082e4db-d513-4905-8eb6-deb2cc6d6272 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.462599 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2082e4db-d513-4905-8eb6-deb2cc6d6272 admin admin] PATH_INFO: `/v3/projects/da749564df7941929c6cb9159479d637/users/04e86369906d4633b02a5ca1bcc24b35/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.477953 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082e4db-d513-4905-8eb6-deb2cc6d6272 admin admin] RBAC: Authorizing `identity:create_grant(project_id=da749564df7941929c6cb9159479d637, user_id=04e86369906d4633b02a5ca1bcc24b35, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.480647 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2082e4db-d513-4905-8eb6-deb2cc6d6272 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.484012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 790/1571] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:06:46 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:46.484804 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464b614-17a1-48f2-8f3f-1133fec3193c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.496695 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 782/1572] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:06:46 2022] PUT /identity/v3/projects/da749564df7941929c6cb9159479d637/users/04e86369906d4633b02a5ca1bcc24b35/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:06:46.498116 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082e4db-d513-4905-8eb6-deb2cc6d6272 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.543790 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:46.556594 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7464b614-17a1-48f2-8f3f-1133fec3193c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.567912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2082e4db-d513-4905-8eb6-deb2cc6d6272 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.573393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80396a27-3e53-4cd5-8a42-7ae5402cc4b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.574070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80396a27-3e53-4cd5-8a42-7ae5402cc4b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.574383 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80396a27-3e53-4cd5-8a42-7ae5402cc4b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.574706 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80396a27-3e53-4cd5-8a42-7ae5402cc4b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.575379 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80396a27-3e53-4cd5-8a42-7ae5402cc4b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.577500 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80396a27-3e53-4cd5-8a42-7ae5402cc4b0 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.585440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:46.606198 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 783/1573] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:06:46 2022] POST /identity/v3/projects => generated 367 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:46.606198 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80396a27-3e53-4cd5-8a42-7ae5402cc4b0 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.624704 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e77357-5705-427a-9c9b-c50cd34c107a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.625480 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e77357-5705-427a-9c9b-c50cd34c107a admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.625771 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e77357-5705-427a-9c9b-c50cd34c107a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.626051 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e77357-5705-427a-9c9b-c50cd34c107a admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.627377 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e77357-5705-427a-9c9b-c50cd34c107a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.629847 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42e77357-5705-427a-9c9b-c50cd34c107a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.653324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 791/1574] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:06:46 2022] POST /identity/v3/users => generated 289 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:46.653324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e77357-5705-427a-9c9b-c50cd34c107a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.663491 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e77357-5705-427a-9c9b-c50cd34c107a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.676687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80396a27-3e53-4cd5-8a42-7ae5402cc4b0 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.682613 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210c36c-5a75-44e3-85e6-f38f2ec4d03c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.683289 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210c36c-5a75-44e3-85e6-f38f2ec4d03c admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.683575 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210c36c-5a75-44e3-85e6-f38f2ec4d03c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.684148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f210c36c-5a75-44e3-85e6-f38f2ec4d03c admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.684847 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210c36c-5a75-44e3-85e6-f38f2ec4d03c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.686813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f210c36c-5a75-44e3-85e6-f38f2ec4d03c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.714429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 784/1575] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:06:46 2022] POST /identity/v3/projects => generated 377 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:46.715389 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210c36c-5a75-44e3-85e6-f38f2ec4d03c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.725308 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d7803-c873-43b5-a66b-530a238d3b6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.725998 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d7803-c873-43b5-a66b-530a238d3b6e admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.726291 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d7803-c873-43b5-a66b-530a238d3b6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.726599 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b5d7803-c873-43b5-a66b-530a238d3b6e admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.727306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d7803-c873-43b5-a66b-530a238d3b6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.729241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b5d7803-c873-43b5-a66b-530a238d3b6e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.749675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 792/1576] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:06:46 2022] POST /identity/v3/users => generated 295 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:46.751248 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d7803-c873-43b5-a66b-530a238d3b6e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.761347 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b5d7803-c873-43b5-a66b-530a238d3b6e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.777270 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f210c36c-5a75-44e3-85e6-f38f2ec4d03c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.782108 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e44ec-5f34-471e-a414-707e2f3ff988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.782916 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e44ec-5f34-471e-a414-707e2f3ff988 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.783232 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e44ec-5f34-471e-a414-707e2f3ff988 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.783954 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9e44ec-5f34-471e-a414-707e2f3ff988 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.784661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9e44ec-5f34-471e-a414-707e2f3ff988 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.785702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: WARNING py.warnings [None req-2c9e44ec-5f34-471e-a414-707e2f3ff988 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:06:46.785702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: warnings.warn(msg) Jul 11 14:06:46.785702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: Jul 11 14:06:46.786937 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9e44ec-5f34-471e-a414-707e2f3ff988 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.808894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 785/1577] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:06:46 2022] POST /identity/v3/users => generated 289 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:46.809716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e44ec-5f34-471e-a414-707e2f3ff988 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.823309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9e44ec-5f34-471e-a414-707e2f3ff988 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.824415 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51258ae0-d51f-43e8-899f-d0c5b0097ae0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.825084 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51258ae0-d51f-43e8-899f-d0c5b0097ae0 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.825376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51258ae0-d51f-43e8-899f-d0c5b0097ae0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.825657 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51258ae0-d51f-43e8-899f-d0c5b0097ae0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.826383 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51258ae0-d51f-43e8-899f-d0c5b0097ae0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.828284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51258ae0-d51f-43e8-899f-d0c5b0097ae0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.830918 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236978c-105d-443f-9665-df3479c8607b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.831612 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236978c-105d-443f-9665-df3479c8607b admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.831894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236978c-105d-443f-9665-df3479c8607b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.832170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236978c-105d-443f-9665-df3479c8607b admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.832849 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236978c-105d-443f-9665-df3479c8607b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.834786 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a236978c-105d-443f-9665-df3479c8607b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.855584 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 793/1578] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:06:46 2022] POST /identity/v3/projects => generated 391 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:46.856759 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 786/1579] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:06:46 2022] POST /identity/v3/users => generated 292 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:46.857165 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863242e7-d65d-4d9d-ad98-e6cd158c4b9b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.857446 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236978c-105d-443f-9665-df3479c8607b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.857626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863242e7-d65d-4d9d-ad98-e6cd158c4b9b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.857887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863242e7-d65d-4d9d-ad98-e6cd158c4b9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.871745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a236978c-105d-443f-9665-df3479c8607b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.876384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-863242e7-d65d-4d9d-ad98-e6cd158c4b9b None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:46.877682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-863242e7-d65d-4d9d-ad98-e6cd158c4b9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:46.889389 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:46.905393 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:46.931992 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509ec73-bbd4-4923-be4f-af1d3bb411f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.932592 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509ec73-bbd4-4923-be4f-af1d3bb411f6 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.932880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509ec73-bbd4-4923-be4f-af1d3bb411f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.933162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f509ec73-bbd4-4923-be4f-af1d3bb411f6 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.933863 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509ec73-bbd4-4923-be4f-af1d3bb411f6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.935893 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f509ec73-bbd4-4923-be4f-af1d3bb411f6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.942773 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 794/1580] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:46.943993 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863242e7-d65d-4d9d-ad98-e6cd158c4b9b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.954863 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863242e7-d65d-4d9d-ad98-e6cd158c4b9b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.959959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af9bfb2-30fd-4ff1-8f64-8b9abb8f2e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.960649 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9bfb2-30fd-4ff1-8f64-8b9abb8f2e48 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.960916 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9bfb2-30fd-4ff1-8f64-8b9abb8f2e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.961194 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af9bfb2-30fd-4ff1-8f64-8b9abb8f2e48 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.964876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 787/1581] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:06:46 2022] POST /identity/v3/users => generated 284 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:46.965018 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af9bfb2-30fd-4ff1-8f64-8b9abb8f2e48 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.965129 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509ec73-bbd4-4923-be4f-af1d3bb411f6 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.965211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af9bfb2-30fd-4ff1-8f64-8b9abb8f2e48 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.972984 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f509ec73-bbd4-4923-be4f-af1d3bb411f6 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.975617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 795/1582] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:06:46.976423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af9bfb2-30fd-4ff1-8f64-8b9abb8f2e48 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:46.977790 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c6310-087e-40c8-beba-5cca5dd0a9ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.978482 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c6310-087e-40c8-beba-5cca5dd0a9ff admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.978763 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c6310-087e-40c8-beba-5cca5dd0a9ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.979039 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9c6310-087e-40c8-beba-5cca5dd0a9ff admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.979722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c6310-087e-40c8-beba-5cca5dd0a9ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.981716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c9c6310-087e-40c8-beba-5cca5dd0a9ff admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:46.987056 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af9bfb2-30fd-4ff1-8f64-8b9abb8f2e48 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:46.992141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e0393-8554-45f5-8127-7b2103c54b3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:46.993143 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e0393-8554-45f5-8127-7b2103c54b3d admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:46.993548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e0393-8554-45f5-8127-7b2103c54b3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:46.993948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a2e0393-8554-45f5-8127-7b2103c54b3d admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:46.995129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e0393-8554-45f5-8127-7b2103c54b3d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:46.996961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a2e0393-8554-45f5-8127-7b2103c54b3d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.002406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 788/1583] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:06:46 2022] POST /identity/v3/users => generated 289 bytes in 40 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:47.003302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c6310-087e-40c8-beba-5cca5dd0a9ff admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.011009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 796/1584] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:06:47.011821 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e0393-8554-45f5-8127-7b2103c54b3d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.012798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9c6310-087e-40c8-beba-5cca5dd0a9ff admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.017506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b6abd-b168-42a0-8502-7daf4b593e5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.018164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b6abd-b168-42a0-8502-7daf4b593e5d admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.018482 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b6abd-b168-42a0-8502-7daf4b593e5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.018760 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f31b6abd-b168-42a0-8502-7daf4b593e5d admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.019488 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b6abd-b168-42a0-8502-7daf4b593e5d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.021086 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f31b6abd-b168-42a0-8502-7daf4b593e5d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.023653 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a2e0393-8554-45f5-8127-7b2103c54b3d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.028461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423246c-1ccd-4ebf-9433-9f51d5e42c0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.029122 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423246c-1ccd-4ebf-9433-9f51d5e42c0a admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.029418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423246c-1ccd-4ebf-9433-9f51d5e42c0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.029694 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7423246c-1ccd-4ebf-9433-9f51d5e42c0a admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.030423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423246c-1ccd-4ebf-9433-9f51d5e42c0a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.032068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 789/1585] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:06:47.033651 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7423246c-1ccd-4ebf-9433-9f51d5e42c0a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.033774 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b6abd-b168-42a0-8502-7daf4b593e5d admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.045423 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f31b6abd-b168-42a0-8502-7daf4b593e5d admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.051357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8662b-43ab-4500-8f3f-62d9b85a498c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.052364 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8662b-43ab-4500-8f3f-62d9b85a498c admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.053281 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8662b-43ab-4500-8f3f-62d9b85a498c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.053281 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da8662b-43ab-4500-8f3f-62d9b85a498c admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.053604 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da8662b-43ab-4500-8f3f-62d9b85a498c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.055286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da8662b-43ab-4500-8f3f-62d9b85a498c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.055940 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 797/1586] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:06:47 2022] POST /identity/v3/users => generated 296 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:47.057079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423246c-1ccd-4ebf-9433-9f51d5e42c0a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.069312 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 790/1587] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:06:47.069312 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8662b-43ab-4500-8f3f-62d9b85a498c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.077134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da8662b-43ab-4500-8f3f-62d9b85a498c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.081906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae61be1-12c7-4132-80b9-c48fd80fa36a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.082604 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae61be1-12c7-4132-80b9-c48fd80fa36a admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.082894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae61be1-12c7-4132-80b9-c48fd80fa36a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.083176 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ae61be1-12c7-4132-80b9-c48fd80fa36a admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.084025 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae61be1-12c7-4132-80b9-c48fd80fa36a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.085515 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ae61be1-12c7-4132-80b9-c48fd80fa36a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.099000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 791/1588] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:06:47.099000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae61be1-12c7-4132-80b9-c48fd80fa36a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.106666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ae61be1-12c7-4132-80b9-c48fd80fa36a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.112061 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2a3a6-c901-4368-a2b4-374e841b31a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.112713 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2a3a6-c901-4368-a2b4-374e841b31a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.113092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2a3a6-c901-4368-a2b4-374e841b31a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.113282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae2a3a6-c901-4368-a2b4-374e841b31a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.114009 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2a3a6-c901-4368-a2b4-374e841b31a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.115649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae2a3a6-c901-4368-a2b4-374e841b31a5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.123449 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7423246c-1ccd-4ebf-9433-9f51d5e42c0a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.126137 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 792/1589] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:06:47.127062 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44928d-3383-4d38-a422-cd795da06993 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.127713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44928d-3383-4d38-a422-cd795da06993 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.128045 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44928d-3383-4d38-a422-cd795da06993 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.128328 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e44928d-3383-4d38-a422-cd795da06993 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.130476 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2a3a6-c901-4368-a2b4-374e841b31a5 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.137677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44928d-3383-4d38-a422-cd795da06993 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.139195 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e44928d-3383-4d38-a422-cd795da06993 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.140598 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae2a3a6-c901-4368-a2b4-374e841b31a5 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.145301 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ebce8-6dd9-4d12-93f3-17a33d66df6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.145958 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ebce8-6dd9-4d12-93f3-17a33d66df6d admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.146244 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ebce8-6dd9-4d12-93f3-17a33d66df6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.146564 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ebce8-6dd9-4d12-93f3-17a33d66df6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.147298 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ebce8-6dd9-4d12-93f3-17a33d66df6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.148887 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ebce8-6dd9-4d12-93f3-17a33d66df6d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.149146 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 798/1590] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:06:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:06:47.150252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44928d-3383-4d38-a422-cd795da06993 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.160015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 793/1591] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:06:47.161259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e44928d-3383-4d38-a422-cd795da06993 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.163852 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ebce8-6dd9-4d12-93f3-17a33d66df6d admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.166214 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2cdc1-faa7-43bd-8c20-644ff8976d73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.166916 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2cdc1-faa7-43bd-8c20-644ff8976d73 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.167338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2cdc1-faa7-43bd-8c20-644ff8976d73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.167628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e2cdc1-faa7-43bd-8c20-644ff8976d73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.168395 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2cdc1-faa7-43bd-8c20-644ff8976d73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.170006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e2cdc1-faa7-43bd-8c20-644ff8976d73 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.176558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ebce8-6dd9-4d12-93f3-17a33d66df6d admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.181676 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 799/1592] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:06:47.181911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27ae56-c878-4d26-bb29-1b6f0f64618f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.182591 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2cdc1-faa7-43bd-8c20-644ff8976d73 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.182733 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27ae56-c878-4d26-bb29-1b6f0f64618f admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.182917 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27ae56-c878-4d26-bb29-1b6f0f64618f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.183381 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae27ae56-c878-4d26-bb29-1b6f0f64618f admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.184117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27ae56-c878-4d26-bb29-1b6f0f64618f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.185791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae27ae56-c878-4d26-bb29-1b6f0f64618f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.196617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e2cdc1-faa7-43bd-8c20-644ff8976d73 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.199903 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 794/1593] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:06:47.199903 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27ae56-c878-4d26-bb29-1b6f0f64618f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.201607 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33408afd-6622-49bf-a023-f3764b139739 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.202274 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33408afd-6622-49bf-a023-f3764b139739 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.202587 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33408afd-6622-49bf-a023-f3764b139739 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.203065 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33408afd-6622-49bf-a023-f3764b139739 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.203800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33408afd-6622-49bf-a023-f3764b139739 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.206048 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33408afd-6622-49bf-a023-f3764b139739 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.208699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae27ae56-c878-4d26-bb29-1b6f0f64618f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.214072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6085a-0f87-46e3-9191-fd6a5567c055 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.214783 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6085a-0f87-46e3-9191-fd6a5567c055 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.215067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6085a-0f87-46e3-9191-fd6a5567c055 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.215726 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5c6085a-0f87-46e3-9191-fd6a5567c055 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.216074 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6085a-0f87-46e3-9191-fd6a5567c055 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.217833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5c6085a-0f87-46e3-9191-fd6a5567c055 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.217959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 800/1594] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:06:47.219696 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33408afd-6622-49bf-a023-f3764b139739 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.232105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33408afd-6622-49bf-a023-f3764b139739 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.232743 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 795/1595] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:06:47.233583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6085a-0f87-46e3-9191-fd6a5567c055 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.238272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa1911-94e3-47b8-8608-fae4b2d1c45d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.238979 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa1911-94e3-47b8-8608-fae4b2d1c45d admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.239635 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa1911-94e3-47b8-8608-fae4b2d1c45d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.239635 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1aa1911-94e3-47b8-8608-fae4b2d1c45d admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.240259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa1911-94e3-47b8-8608-fae4b2d1c45d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.241941 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1aa1911-94e3-47b8-8608-fae4b2d1c45d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.246078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5c6085a-0f87-46e3-9191-fd6a5567c055 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.251133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089eb48-22ec-4aba-b827-b69a08cd7d04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.252199 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089eb48-22ec-4aba-b827-b69a08cd7d04 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.252511 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089eb48-22ec-4aba-b827-b69a08cd7d04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.252791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0089eb48-22ec-4aba-b827-b69a08cd7d04 admin admin] PATH_INFO: `/v3/projects/3b9e53c9298348959c10d9e79a7a5e63/users/c3d3348c338b4567bfea396fa7645a17/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.254100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 801/1596] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:06:47.257904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa1911-94e3-47b8-8608-fae4b2d1c45d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.268308 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1aa1911-94e3-47b8-8608-fae4b2d1c45d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.270471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089eb48-22ec-4aba-b827-b69a08cd7d04 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b9e53c9298348959c10d9e79a7a5e63, user_id=c3d3348c338b4567bfea396fa7645a17, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.273314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5084bc2-8568-4197-a1e1-1ca37d2fa888 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.273506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0089eb48-22ec-4aba-b827-b69a08cd7d04 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.273977 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5084bc2-8568-4197-a1e1-1ca37d2fa888 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.274269 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5084bc2-8568-4197-a1e1-1ca37d2fa888 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.274580 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5084bc2-8568-4197-a1e1-1ca37d2fa888 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.275311 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5084bc2-8568-4197-a1e1-1ca37d2fa888 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.276891 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5084bc2-8568-4197-a1e1-1ca37d2fa888 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.288897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 796/1597] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:06:47 2022] PUT /identity/v3/projects/3b9e53c9298348959c10d9e79a7a5e63/users/c3d3348c338b4567bfea396fa7645a17/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:06:47.289340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 802/1598] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:06:47.289707 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089eb48-22ec-4aba-b827-b69a08cd7d04 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.290220 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5084bc2-8568-4197-a1e1-1ca37d2fa888 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.302767 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5084bc2-8568-4197-a1e1-1ca37d2fa888 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.349419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:47.363823 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:47.368906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0089eb48-22ec-4aba-b827-b69a08cd7d04 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.375663 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bff81-aa06-4410-b95c-153836977a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.377140 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bff81-aa06-4410-b95c-153836977a2e admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.377439 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bff81-aa06-4410-b95c-153836977a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.377780 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423bff81-aa06-4410-b95c-153836977a2e admin admin] PATH_INFO: `/v3/projects/beef80716d254a03a5e673b9cfd781ce/users/98143abd2e3c47e3921184a1c5e05a89/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.381785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76869f-fca3-4147-9534-f8eca35fd2c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.382877 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76869f-fca3-4147-9534-f8eca35fd2c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.383171 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76869f-fca3-4147-9534-f8eca35fd2c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.383487 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a76869f-fca3-4147-9534-f8eca35fd2c1 admin admin] PATH_INFO: `/v3/projects/a772cbaa461c416da2cc35e364541ecf/users/296853a88430438e96e0f8fb97b46d2e/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.396381 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bff81-aa06-4410-b95c-153836977a2e admin admin] RBAC: Authorizing `identity:create_grant(project_id=beef80716d254a03a5e673b9cfd781ce, user_id=98143abd2e3c47e3921184a1c5e05a89, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.399535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423bff81-aa06-4410-b95c-153836977a2e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.403182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76869f-fca3-4147-9534-f8eca35fd2c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a772cbaa461c416da2cc35e364541ecf, user_id=296853a88430438e96e0f8fb97b46d2e, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.404241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-2a76869f-fca3-4147-9534-f8eca35fd2c1 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:06:47.404241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: warnings.warn(msg) Jul 11 14:06:47.404241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: Jul 11 14:06:47.407612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a76869f-fca3-4147-9534-f8eca35fd2c1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.422367 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 797/1599] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:06:47 2022] PUT /identity/v3/projects/beef80716d254a03a5e673b9cfd781ce/users/98143abd2e3c47e3921184a1c5e05a89/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:06:47.422367 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bff81-aa06-4410-b95c-153836977a2e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.438568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 803/1600] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:06:47 2022] PUT /identity/v3/projects/a772cbaa461c416da2cc35e364541ecf/users/296853a88430438e96e0f8fb97b46d2e/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:06:47.438568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76869f-fca3-4147-9534-f8eca35fd2c1 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.512029 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423bff81-aa06-4410-b95c-153836977a2e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.519371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae10e49-5bda-4b95-83fd-4f8ef167dcb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.520367 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae10e49-5bda-4b95-83fd-4f8ef167dcb5 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.520507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a76869f-fca3-4147-9534-f8eca35fd2c1 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.520924 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae10e49-5bda-4b95-83fd-4f8ef167dcb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.520924 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae10e49-5bda-4b95-83fd-4f8ef167dcb5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.521732 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae10e49-5bda-4b95-83fd-4f8ef167dcb5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.523433 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae10e49-5bda-4b95-83fd-4f8ef167dcb5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.528048 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbdec6-9c53-4dbd-a7aa-2af9ef8cee31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.530154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbdec6-9c53-4dbd-a7aa-2af9ef8cee31 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.530962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbdec6-9c53-4dbd-a7aa-2af9ef8cee31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.531747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ecbdec6-9c53-4dbd-a7aa-2af9ef8cee31 admin admin] PATH_INFO: `/v3/projects/15fd934ea11040fe93f64b06bb1e4e21/users/85a11b01e8dd4f1e9b83c9965853a99b/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.534955 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 798/1601] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:47 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:06:47.535786 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae10e49-5bda-4b95-83fd-4f8ef167dcb5 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.548261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae10e49-5bda-4b95-83fd-4f8ef167dcb5 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.555749 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbdec6-9c53-4dbd-a7aa-2af9ef8cee31 admin admin] RBAC: Authorizing `identity:create_grant(project_id=15fd934ea11040fe93f64b06bb1e4e21, user_id=85a11b01e8dd4f1e9b83c9965853a99b, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.558565 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d496c87b-fa42-40ae-a0b0-c8cc443196dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.559447 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ecbdec6-9c53-4dbd-a7aa-2af9ef8cee31 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.559822 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496c87b-fa42-40ae-a0b0-c8cc443196dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.560209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496c87b-fa42-40ae-a0b0-c8cc443196dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.560593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d496c87b-fa42-40ae-a0b0-c8cc443196dc admin admin] PATH_INFO: `/v3/projects/2c46048d0ca34522984ed0eeb26e8189/users/31122814faeb474b84542c8cf6bd26c1/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.580808 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496c87b-fa42-40ae-a0b0-c8cc443196dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c46048d0ca34522984ed0eeb26e8189, user_id=31122814faeb474b84542c8cf6bd26c1, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.581808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 804/1602] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:06:47 2022] PUT /identity/v3/projects/15fd934ea11040fe93f64b06bb1e4e21/users/85a11b01e8dd4f1e9b83c9965853a99b/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:06:47.583234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbdec6-9c53-4dbd-a7aa-2af9ef8cee31 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.588749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d496c87b-fa42-40ae-a0b0-c8cc443196dc admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.610482 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 799/1603] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:06:47 2022] PUT /identity/v3/projects/2c46048d0ca34522984ed0eeb26e8189/users/31122814faeb474b84542c8cf6bd26c1/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:06:47.612622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab4412-32c6-4565-aeea-7c5fc591ceae None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.613378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab4412-32c6-4565-aeea-7c5fc591ceae None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.613969 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ab4412-32c6-4565-aeea-7c5fc591ceae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.644667 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e3ab4412-32c6-4565-aeea-7c5fc591ceae None None] MFA Rules not processed for user `c3d3348c338b4567bfea396fa7645a17`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:47.646163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e3ab4412-32c6-4565-aeea-7c5fc591ceae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:47.677467 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ecbdec6-9c53-4dbd-a7aa-2af9ef8cee31 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.727970 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 800/1604] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:47 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:47.729125 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2ecd3-ae46-4704-89eb-f9f3e35d56a0 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.729481 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2ecd3-ae46-4704-89eb-f9f3e35d56a0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.730197 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dba2ecd3-ae46-4704-89eb-f9f3e35d56a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.748912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-dba2ecd3-ae46-4704-89eb-f9f3e35d56a0 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:47.750148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dba2ecd3-ae46-4704-89eb-f9f3e35d56a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:47.758876 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c54c15-638e-4b26-bc01-d5383fe4229b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.759790 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c54c15-638e-4b26-bc01-d5383fe4229b admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.760076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c54c15-638e-4b26-bc01-d5383fe4229b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.760357 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c54c15-638e-4b26-bc01-d5383fe4229b admin admin] PATH_INFO: `/v3/projects/7136b6633c6e4d399a6d359d750125f0/users/170f249dccd4498990a8344a18c73059/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.770103 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:47.780714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c54c15-638e-4b26-bc01-d5383fe4229b admin admin] RBAC: Authorizing `identity:create_grant(project_id=7136b6633c6e4d399a6d359d750125f0, user_id=170f249dccd4498990a8344a18c73059, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.784692 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c54c15-638e-4b26-bc01-d5383fe4229b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.789425 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:47.800692 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 805/1605] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:06:47 2022] PUT /identity/v3/projects/7136b6633c6e4d399a6d359d750125f0/users/170f249dccd4498990a8344a18c73059/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:06:47.805714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26c561-376b-4782-817b-97eb0ab3284c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.806082 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26c561-376b-4782-817b-97eb0ab3284c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.806463 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d26c561-376b-4782-817b-97eb0ab3284c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.824116 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 801/1606] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:47.825291 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2ecd3-ae46-4704-89eb-f9f3e35d56a0 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:47.827127 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8d26c561-376b-4782-817b-97eb0ab3284c None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:47.829166 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8d26c561-376b-4782-817b-97eb0ab3284c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:47.837323 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dba2ecd3-ae46-4704-89eb-f9f3e35d56a0 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:47.901791 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 806/1607] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:47.903426 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0219178-228e-4776-8946-29bc7891233d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.904057 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0219178-228e-4776-8946-29bc7891233d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.904346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0219178-228e-4776-8946-29bc7891233d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.908504 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12b9e4fc-4e3f-4036-8d36-08cc93cb0da6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:47.910759 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9e4fc-4e3f-4036-8d36-08cc93cb0da6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.911139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9e4fc-4e3f-4036-8d36-08cc93cb0da6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.911432 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12b9e4fc-4e3f-4036-8d36-08cc93cb0da6 admin admin] PATH_INFO: `/v3/projects/4b181bc391914f6abcea09b894b88473/users/cf6cc756845148ee8fce9021425cb7cb/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:47.928869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9e4fc-4e3f-4036-8d36-08cc93cb0da6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b181bc391914f6abcea09b894b88473, user_id=cf6cc756845148ee8fce9021425cb7cb, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:47.931681 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12b9e4fc-4e3f-4036-8d36-08cc93cb0da6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:47.936260 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d0219178-228e-4776-8946-29bc7891233d None None] MFA Rules not processed for user `85a11b01e8dd4f1e9b83c9965853a99b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:47.937036 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d0219178-228e-4776-8946-29bc7891233d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:47.964123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 802/1608] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:06:47 2022] PUT /identity/v3/projects/4b181bc391914f6abcea09b894b88473/users/cf6cc756845148ee8fce9021425cb7cb/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:06:47.966039 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d24c6c-4aa1-481b-905f-0861878c3c5c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:47.966369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d24c6c-4aa1-481b-905f-0861878c3c5c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:47.966657 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d24c6c-4aa1-481b-905f-0861878c3c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:48.001077 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-82d24c6c-4aa1-481b-905f-0861878c3c5c None None] MFA Rules not processed for user `31122814faeb474b84542c8cf6bd26c1`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:48.003269 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-82d24c6c-4aa1-481b-905f-0861878c3c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:48.023721 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 807/1609] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:47 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:48.024826 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0219178-228e-4776-8946-29bc7891233d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:48.054733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0219178-228e-4776-8946-29bc7891233d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:48.116571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 803/1610] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:47 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:48.118457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d457c4-1c95-4d49-8d0b-238cabfd8287 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:48.118912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d457c4-1c95-4d49-8d0b-238cabfd8287 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:48.119335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d457c4-1c95-4d49-8d0b-238cabfd8287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:48.137416 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc5eb7-2ffb-414a-a882-f0218c13ceb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:48.138465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc5eb7-2ffb-414a-a882-f0218c13ceb9 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:48.138759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc5eb7-2ffb-414a-a882-f0218c13ceb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:48.139038 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59dc5eb7-2ffb-414a-a882-f0218c13ceb9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:48.139800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc5eb7-2ffb-414a-a882-f0218c13ceb9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:48.143832 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59dc5eb7-2ffb-414a-a882-f0218c13ceb9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:48.144440 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-d0d457c4-1c95-4d49-8d0b-238cabfd8287 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:48.146075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d0d457c4-1c95-4d49-8d0b-238cabfd8287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:48.155826 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 804/1611] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:48.156785 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d457c4-1c95-4d49-8d0b-238cabfd8287 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:48.185602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 808/1612] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:06:48 2022] POST /identity/v3/projects => generated 381 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:48.185602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc5eb7-2ffb-414a-a882-f0218c13ceb9 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:48.189077 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:48.213646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:48.226721 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d457c4-1c95-4d49-8d0b-238cabfd8287 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:48.259716 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59dc5eb7-2ffb-414a-a882-f0218c13ceb9 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:48.263755 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-698f068a-d535-43cf-a26a-057cf95f66e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:48.264215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f068a-d535-43cf-a26a-057cf95f66e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:48.264504 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f068a-d535-43cf-a26a-057cf95f66e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:48.264784 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698f068a-d535-43cf-a26a-057cf95f66e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:48.295108 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24bb73ca-0faf-4231-8d5d-f94116d50bce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:48.295828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb73ca-0faf-4231-8d5d-f94116d50bce service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:48.296444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb73ca-0faf-4231-8d5d-f94116d50bce service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:48.296444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24bb73ca-0faf-4231-8d5d-f94116d50bce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:48.364372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f068a-d535-43cf-a26a-057cf95f66e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:48.366745 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-698f068a-d535-43cf-a26a-057cf95f66e9 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:48.383164 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 809/1613] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:06:48.384445 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb73ca-0faf-4231-8d5d-f94116d50bce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:48.385299 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db598d-f958-4619-8048-a96eecc076be None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:48.385604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db598d-f958-4619-8048-a96eecc076be None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:48.385888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9db598d-f958-4619-8048-a96eecc076be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:48.386869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24bb73ca-0faf-4231-8d5d-f94116d50bce service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:48.400757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 805/1614] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:06:48.403814 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329a498-309a-456d-b1f8-fa663e667f4a None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:48.404188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329a498-309a-456d-b1f8-fa663e667f4a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:48.404549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7329a498-309a-456d-b1f8-fa663e667f4a None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:48.411280 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d9db598d-f958-4619-8048-a96eecc076be None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:48.411647 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 806/1615] 10.222.0.49 () {54 vars in 961 bytes} [Mon Jul 11 14:06:48 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:06:48.412700 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329a498-309a-456d-b1f8-fa663e667f4a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:48.414828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d9db598d-f958-4619-8048-a96eecc076be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:48.431196 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 810/1616] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:48.432059 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db598d-f958-4619-8048-a96eecc076be None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:48.436247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d9db598d-f958-4619-8048-a96eecc076be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:48.441340 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7329a498-309a-456d-b1f8-fa663e667f4a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:48.451629 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126b755-c5dc-4af0-b508-978e5d617bb2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:48.452300 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126b755-c5dc-4af0-b508-978e5d617bb2 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:48.452596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126b755-c5dc-4af0-b508-978e5d617bb2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:48.452858 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9126b755-c5dc-4af0-b508-978e5d617bb2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:48.453554 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9126b755-c5dc-4af0-b508-978e5d617bb2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:48.456183 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9126b755-c5dc-4af0-b508-978e5d617bb2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:48.461641 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9db598d-f958-4619-8048-a96eecc076be None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:48.466577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5f9ce33-64fb-421a-9a3f-2467fc044bb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:48.468664 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9ce33-64fb-421a-9a3f-2467fc044bb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:48.469464 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9ce33-64fb-421a-9a3f-2467fc044bb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:48.469931 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5f9ce33-64fb-421a-9a3f-2467fc044bb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:48.489576 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9ce33-64fb-421a-9a3f-2467fc044bb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:48.491718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 807/1617] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:06:48 2022] POST /identity/v3/projects => generated 369 bytes in 79 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:48.492005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5f9ce33-64fb-421a-9a3f-2467fc044bb7 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:48.492882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126b755-c5dc-4af0-b508-978e5d617bb2 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:48.510532 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 811/1618] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:06:48.512292 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3d3b3-84b8-41bd-a708-c022d605f626 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:48.512596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3d3b3-84b8-41bd-a708-c022d605f626 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:48.512880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b3d3b3-84b8-41bd-a708-c022d605f626 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:48.566335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a0b3d3b3-84b8-41bd-a708-c022d605f626 None None] MFA Rules not processed for user `ef5dac3f311e43809a46995bab56e780`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:48.567616 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a0b3d3b3-84b8-41bd-a708-c022d605f626 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:48.595856 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9126b755-c5dc-4af0-b508-978e5d617bb2 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:48.602532 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fbf7f-2419-473c-8b4f-aefc826f3a2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:48.603514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fbf7f-2419-473c-8b4f-aefc826f3a2f admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:48.603872 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fbf7f-2419-473c-8b4f-aefc826f3a2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:48.604222 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4fbf7f-2419-473c-8b4f-aefc826f3a2f admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:48.605000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fbf7f-2419-473c-8b4f-aefc826f3a2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:48.607260 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4fbf7f-2419-473c-8b4f-aefc826f3a2f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:48.659732 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 808/1619] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:06:48 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:48.660770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fbf7f-2419-473c-8b4f-aefc826f3a2f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:48.710773 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:48.741378 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 812/1620] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Jul 11 14:06:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 230 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:48.743442 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3d3b3-84b8-41bd-a708-c022d605f626 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:48.766498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b3d3b3-84b8-41bd-a708-c022d605f626 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:48.780640 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125cacc-e4c6-42c2-a552-3c1c29482d9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:48.781312 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125cacc-e4c6-42c2-a552-3c1c29482d9f admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:48.781602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125cacc-e4c6-42c2-a552-3c1c29482d9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:48.784074 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c125cacc-e4c6-42c2-a552-3c1c29482d9f admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:48.784952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125cacc-e4c6-42c2-a552-3c1c29482d9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:48.787067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c125cacc-e4c6-42c2-a552-3c1c29482d9f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:48.808814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:48.810758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4fbf7f-2419-473c-8b4f-aefc826f3a2f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:48.816131 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40142d-7cef-40b0-86e0-70e6547e0eb4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:48.816946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40142d-7cef-40b0-86e0-70e6547e0eb4 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:48.817162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40142d-7cef-40b0-86e0-70e6547e0eb4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:48.817784 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b40142d-7cef-40b0-86e0-70e6547e0eb4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:48.834130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40142d-7cef-40b0-86e0-70e6547e0eb4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:48.836712 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b40142d-7cef-40b0-86e0-70e6547e0eb4 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:48.842184 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 813/1621] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:06:48 2022] POST /identity/v3/users => generated 291 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:06:48.843659 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125cacc-e4c6-42c2-a552-3c1c29482d9f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:48.852953 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 809/1622] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:06:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:06:48.854861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40142d-7cef-40b0-86e0-70e6547e0eb4 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:48.870474 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c125cacc-e4c6-42c2-a552-3c1c29482d9f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:48.871962 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b40142d-7cef-40b0-86e0-70e6547e0eb4 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:48.880046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b496b-d0af-491f-9a07-0182764c7515 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:48.880828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b496b-d0af-491f-9a07-0182764c7515 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:48.881184 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b496b-d0af-491f-9a07-0182764c7515 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:48.881763 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b496b-d0af-491f-9a07-0182764c7515 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:48.884500 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb388f1-a6ec-42ea-bf56-220080ecc5a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:48.885232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb388f1-a6ec-42ea-bf56-220080ecc5a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:48.885523 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb388f1-a6ec-42ea-bf56-220080ecc5a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:48.885802 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb388f1-a6ec-42ea-bf56-220080ecc5a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:48.888557 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb388f1-a6ec-42ea-bf56-220080ecc5a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:48.891680 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fb388f1-a6ec-42ea-bf56-220080ecc5a4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:48.916799 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 814/1623] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:06:48.925517 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb388f1-a6ec-42ea-bf56-220080ecc5a4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:48.950810 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fb388f1-a6ec-42ea-bf56-220080ecc5a4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:48.959503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b401a-a0e9-4aeb-94ce-62ee786d2a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:48.960936 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b401a-a0e9-4aeb-94ce-62ee786d2a52 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:48.961496 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b401a-a0e9-4aeb-94ce-62ee786d2a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:48.962079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458b401a-a0e9-4aeb-94ce-62ee786d2a52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:48.963126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b401a-a0e9-4aeb-94ce-62ee786d2a52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:48.965779 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-458b401a-a0e9-4aeb-94ce-62ee786d2a52 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:48.984387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 815/1624] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:48 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:06:48.985508 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b401a-a0e9-4aeb-94ce-62ee786d2a52 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:48.986784 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b496b-d0af-491f-9a07-0182764c7515 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:48.989122 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b496b-d0af-491f-9a07-0182764c7515 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:48.998900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458b401a-a0e9-4aeb-94ce-62ee786d2a52 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:49.004763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649f34d-65c6-457a-899a-3c6fa7537bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:49.006074 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649f34d-65c6-457a-899a-3c6fa7537bf2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:49.006847 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649f34d-65c6-457a-899a-3c6fa7537bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:49.007357 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6649f34d-65c6-457a-899a-3c6fa7537bf2 admin admin] PATH_INFO: `/v3/projects/8aec473535ea4a208e41a21d422ed1ec/users/76e1951a4e014d1ca1fabe64c0792285/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:49.016080 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 810/1625] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:06:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:06:49.019639 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b496b-d0af-491f-9a07-0182764c7515 service nova] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:49.029063 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649f34d-65c6-457a-899a-3c6fa7537bf2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aec473535ea4a208e41a21d422ed1ec, user_id=76e1951a4e014d1ca1fabe64c0792285, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:49.032729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6649f34d-65c6-457a-899a-3c6fa7537bf2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:49.058184 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b496b-d0af-491f-9a07-0182764c7515 service nova] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:49.066680 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 816/1626] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:06:48 2022] PUT /identity/v3/projects/8aec473535ea4a208e41a21d422ed1ec/users/76e1951a4e014d1ca1fabe64c0792285/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:06:49.076658 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649f34d-65c6-457a-899a-3c6fa7537bf2 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:49.196896 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c960a-ee2c-4059-8fff-73dc6847d7e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:49.198039 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c960a-ee2c-4059-8fff-73dc6847d7e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:49.198415 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c960a-ee2c-4059-8fff-73dc6847d7e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:49.198702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412c960a-ee2c-4059-8fff-73dc6847d7e4 admin admin] PATH_INFO: `/v3/projects/5d9bfa59e0e840f1bebaeb5042d54a01/users/df4f904f36034cdb9bc4e7b28b6036b7/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:49.232243 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c960a-ee2c-4059-8fff-73dc6847d7e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d9bfa59e0e840f1bebaeb5042d54a01, user_id=df4f904f36034cdb9bc4e7b28b6036b7, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:49.236376 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-412c960a-ee2c-4059-8fff-73dc6847d7e4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:49.244108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6649f34d-65c6-457a-899a-3c6fa7537bf2 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:49.260553 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57cd12-25a5-405d-aec8-d451e5242fb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:49.262798 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57cd12-25a5-405d-aec8-d451e5242fb0 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:49.263100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57cd12-25a5-405d-aec8-d451e5242fb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:49.263385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f57cd12-25a5-405d-aec8-d451e5242fb0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:49.264561 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57cd12-25a5-405d-aec8-d451e5242fb0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:49.266376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f57cd12-25a5-405d-aec8-d451e5242fb0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:49.274184 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 811/1627] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:06:49 2022] PUT /identity/v3/projects/5d9bfa59e0e840f1bebaeb5042d54a01/users/df4f904f36034cdb9bc4e7b28b6036b7/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:06:49.277629 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c960a-ee2c-4059-8fff-73dc6847d7e4 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:49.279332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 817/1628] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:06:49.284348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57cd12-25a5-405d-aec8-d451e5242fb0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:49.319800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f57cd12-25a5-405d-aec8-d451e5242fb0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:49.370805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:49.444127 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412c960a-ee2c-4059-8fff-73dc6847d7e4 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:49.447837 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56750e69-c451-4aa3-b838-41cec78a2bd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:49.449148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56750e69-c451-4aa3-b838-41cec78a2bd5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:49.451827 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56750e69-c451-4aa3-b838-41cec78a2bd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:49.452372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56750e69-c451-4aa3-b838-41cec78a2bd5 admin admin] PATH_INFO: `/v3/projects/8aec473535ea4a208e41a21d422ed1ec/users/76e1951a4e014d1ca1fabe64c0792285/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:49.455668 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56750e69-c451-4aa3-b838-41cec78a2bd5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8aec473535ea4a208e41a21d422ed1ec, user_id=76e1951a4e014d1ca1fabe64c0792285, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:49.460615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56750e69-c451-4aa3-b838-41cec78a2bd5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:49.464927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e89fe2f1-684a-4f07-98f6-69a02a6cb037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:49.465605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fe2f1-684a-4f07-98f6-69a02a6cb037 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:49.466284 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fe2f1-684a-4f07-98f6-69a02a6cb037 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:49.466598 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e89fe2f1-684a-4f07-98f6-69a02a6cb037 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:49.467341 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89fe2f1-684a-4f07-98f6-69a02a6cb037 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:49.469331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e89fe2f1-684a-4f07-98f6-69a02a6cb037 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:49.533884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 818/1629] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:06:49 2022] PUT /identity/v3/projects/8aec473535ea4a208e41a21d422ed1ec/users/76e1951a4e014d1ca1fabe64c0792285/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:06:49.533884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fcecf-cb33-4351-b13b-d88ce2789166 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:49.533884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fcecf-cb33-4351-b13b-d88ce2789166 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:49.533884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-876fcecf-cb33-4351-b13b-d88ce2789166 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:49.533884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 819/1630] 10.222.0.49 () {54 vars in 912 bytes} [Mon Jul 11 14:06:49 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:06:49.533884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68117444-bb06-4747-b7fc-6ffa53f5a8a4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:49.562749 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68117444-bb06-4747-b7fc-6ffa53f5a8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:49.562749 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68117444-bb06-4747-b7fc-6ffa53f5a8a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:49.563039 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 812/1631] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:06:49 2022] GET /identity/v3/roles => generated 1420 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:06:49.563039 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc74976-9d7e-46e2-be61-bfc8b173cd34 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:49.563039 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc74976-9d7e-46e2-be61-bfc8b173cd34 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:49.563039 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dc74976-9d7e-46e2-be61-bfc8b173cd34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:49.590542 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-68117444-bb06-4747-b7fc-6ffa53f5a8a4 None None] MFA Rules not processed for user `76e1951a4e014d1ca1fabe64c0792285`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:49.592187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-68117444-bb06-4747-b7fc-6ffa53f5a8a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:49.593646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3dc74976-9d7e-46e2-be61-bfc8b173cd34 None None] MFA Rules not processed for user `ef5dac3f311e43809a46995bab56e780`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:49.596419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3dc74976-9d7e-46e2-be61-bfc8b173cd34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:49.652029 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:49.710933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 813/1632] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 11 14:06:49 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:49.719164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc74976-9d7e-46e2-be61-bfc8b173cd34 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:49.804254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 820/1633] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:49 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:49.804254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68117444-bb06-4747-b7fc-6ffa53f5a8a4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:49.862649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dc74976-9d7e-46e2-be61-bfc8b173cd34 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:49.874538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f8746-1691-435a-9c39-eca80e2d5a7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:49.876290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8746-1691-435a-9c39-eca80e2d5a7e admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:49.876721 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8746-1691-435a-9c39-eca80e2d5a7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:49.877968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-220f8746-1691-435a-9c39-eca80e2d5a7e admin admin] PATH_INFO: `/v3/projects/5d9bfa59e0e840f1bebaeb5042d54a01/users/df4f904f36034cdb9bc4e7b28b6036b7/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:49.886980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f8746-1691-435a-9c39-eca80e2d5a7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=5d9bfa59e0e840f1bebaeb5042d54a01, user_id=df4f904f36034cdb9bc4e7b28b6036b7, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:49.889388 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-220f8746-1691-435a-9c39-eca80e2d5a7e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:49.918508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 814/1634] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:06:49 2022] PUT /identity/v3/projects/5d9bfa59e0e840f1bebaeb5042d54a01/users/df4f904f36034cdb9bc4e7b28b6036b7/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:06:49.919336 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f8746-1691-435a-9c39-eca80e2d5a7e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:49.953104 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68117444-bb06-4747-b7fc-6ffa53f5a8a4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:50.026993 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96048b7e-0138-4d8e-b20e-bfc6a86ce579 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:50.027962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96048b7e-0138-4d8e-b20e-bfc6a86ce579 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:50.028251 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96048b7e-0138-4d8e-b20e-bfc6a86ce579 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:50.029110 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96048b7e-0138-4d8e-b20e-bfc6a86ce579 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:50.085202 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-220f8746-1691-435a-9c39-eca80e2d5a7e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:50.099826 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86942923-d3ee-4d4e-9511-04257b366485 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:50.100379 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86942923-d3ee-4d4e-9511-04257b366485 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:50.100726 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86942923-d3ee-4d4e-9511-04257b366485 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:50.101013 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86942923-d3ee-4d4e-9511-04257b366485 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:50.143031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96048b7e-0138-4d8e-b20e-bfc6a86ce579 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:50.144544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96048b7e-0138-4d8e-b20e-bfc6a86ce579 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:50.151082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:50.171760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 821/1635] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:06:49 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 371 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:06:50.174101 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd82704-e51d-4294-b744-c462d83a6fd4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:50.175535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd82704-e51d-4294-b744-c462d83a6fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:50.175786 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bd82704-e51d-4294-b744-c462d83a6fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:50.241087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-8bd82704-e51d-4294-b744-c462d83a6fd4 None None] MFA Rules not processed for user `df4f904f36034cdb9bc4e7b28b6036b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:50.242651 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8bd82704-e51d-4294-b744-c462d83a6fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:50.262985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:50.270544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86942923-d3ee-4d4e-9511-04257b366485 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:50.272033 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86942923-d3ee-4d4e-9511-04257b366485 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:50.307548 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 815/1636] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:06:49 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 388 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:06:50.441533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 822/1637] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:50 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 269 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:50.534377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86942923-d3ee-4d4e-9511-04257b366485 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:50.548795 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86942923-d3ee-4d4e-9511-04257b366485 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:50.559392 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2696f-0880-42fc-ac41-79ab64842cd3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:50.560331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2696f-0880-42fc-ac41-79ab64842cd3 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:50.560770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2696f-0880-42fc-ac41-79ab64842cd3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:50.561207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e2696f-0880-42fc-ac41-79ab64842cd3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:50.590851 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2696f-0880-42fc-ac41-79ab64842cd3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:50.592493 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e2696f-0880-42fc-ac41-79ab64842cd3 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:50.614903 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 816/1638] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:06:50 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:06:50.696384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd82704-e51d-4294-b744-c462d83a6fd4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:50.711853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bd82704-e51d-4294-b744-c462d83a6fd4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:50.721539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4898646-e03b-4b0a-a8a9-765f66abe1a3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:50.724161 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4898646-e03b-4b0a-a8a9-765f66abe1a3 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:50.725350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4898646-e03b-4b0a-a8a9-765f66abe1a3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:50.726276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4898646-e03b-4b0a-a8a9-765f66abe1a3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:50.814824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4898646-e03b-4b0a-a8a9-765f66abe1a3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:50.817622 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4898646-e03b-4b0a-a8a9-765f66abe1a3 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:50.848277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 823/1639] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:06:50 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:06:51.080816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2696f-0880-42fc-ac41-79ab64842cd3 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:51.101274 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e2696f-0880-42fc-ac41-79ab64842cd3 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:51.112653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83c4fd0-8e96-492a-b661-adb9eb207a61 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:51.114900 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c4fd0-8e96-492a-b661-adb9eb207a61 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:51.115234 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c4fd0-8e96-492a-b661-adb9eb207a61 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:51.115525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83c4fd0-8e96-492a-b661-adb9eb207a61 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:51.288471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c4fd0-8e96-492a-b661-adb9eb207a61 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:51.290263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83c4fd0-8e96-492a-b661-adb9eb207a61 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:51.330153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 817/1640] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:06:51 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 250 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:06:51.770875 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df92cd1-c193-4acc-bef0-7f00335daf10 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:51.771874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df92cd1-c193-4acc-bef0-7f00335daf10 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:51.772379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0df92cd1-c193-4acc-bef0-7f00335daf10 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:51.774110 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 824/1641] 10.222.0.49 () {54 vars in 961 bytes} [Mon Jul 11 14:06:51 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:06:51.796312 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d39f2-1b09-4fd6-9d62-f8284383a3b2 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:51.797422 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d39f2-1b09-4fd6-9d62-f8284383a3b2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:51.798328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57d39f2-1b09-4fd6-9d62-f8284383a3b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:51.828510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b57d39f2-1b09-4fd6-9d62-f8284383a3b2 None None] MFA Rules not processed for user `ef5dac3f311e43809a46995bab56e780`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:51.830817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b57d39f2-1b09-4fd6-9d62-f8284383a3b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:51.863024 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 818/1642] 10.222.0.49 () {58 vars in 1058 bytes} [Mon Jul 11 14:06:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:51.868543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907f594-f855-4eaf-8ebb-fe4f2074981f None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:51.868835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907f594-f855-4eaf-8ebb-fe4f2074981f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:51.869119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4907f594-f855-4eaf-8ebb-fe4f2074981f None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:51.870857 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 825/1643] 10.222.0.49 () {54 vars in 912 bytes} [Mon Jul 11 14:06:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:06:51.876939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8a057-b9b4-4657-a269-33fa6faf0c60 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:51.877480 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8a057-b9b4-4657-a269-33fa6faf0c60 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:51.878403 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cd8a057-b9b4-4657-a269-33fa6faf0c60 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:51.909828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-2cd8a057-b9b4-4657-a269-33fa6faf0c60 None None] MFA Rules not processed for user `ef5dac3f311e43809a46995bab56e780`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:51.912064 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2cd8a057-b9b4-4657-a269-33fa6faf0c60 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:51.937128 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 819/1644] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 11 14:06:51 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 61 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:06:51.950854 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907f594-f855-4eaf-8ebb-fe4f2074981f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:51.971015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4907f594-f855-4eaf-8ebb-fe4f2074981f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:51.988362 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba393e35-dcf5-4689-a927-feaa0baea111 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:51.989245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba393e35-dcf5-4689-a927-feaa0baea111 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:51.989495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba393e35-dcf5-4689-a927-feaa0baea111 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:51.989760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba393e35-dcf5-4689-a927-feaa0baea111 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:52.024200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba393e35-dcf5-4689-a927-feaa0baea111 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:52.027560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba393e35-dcf5-4689-a927-feaa0baea111 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:52.064074 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 826/1645] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:06:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:06:56.740318 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8a057-b9b4-4657-a269-33fa6faf0c60 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:56.895570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cd8a057-b9b4-4657-a269-33fa6faf0c60 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:56.901700 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f6214-d5db-42ee-85ca-ac4adfcda8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:56.902634 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f6214-d5db-42ee-85ca-ac4adfcda8e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:56.902916 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f6214-d5db-42ee-85ca-ac4adfcda8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:56.903197 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33f6214-d5db-42ee-85ca-ac4adfcda8e8 admin admin] PATH_INFO: `/v3/users/31122814faeb474b84542c8cf6bd26c1` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:56.908553 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f6214-d5db-42ee-85ca-ac4adfcda8e8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=31122814faeb474b84542c8cf6bd26c1)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:56.911558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33f6214-d5db-42ee-85ca-ac4adfcda8e8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:56.992224 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:06:57.037647 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c33f6214-d5db-42ee-85ca-ac4adfcda8e8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31122814faeb474b84542c8cf6bd26c1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:06:57.086140 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c33f6214-d5db-42ee-85ca-ac4adfcda8e8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31122814faeb474b84542c8cf6bd26c1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:06:57.107753 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c33f6214-d5db-42ee-85ca-ac4adfcda8e8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31122814faeb474b84542c8cf6bd26c1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:06:57.108698 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c33f6214-d5db-42ee-85ca-ac4adfcda8e8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31122814faeb474b84542c8cf6bd26c1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:06:57.131113 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c33f6214-d5db-42ee-85ca-ac4adfcda8e8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31122814faeb474b84542c8cf6bd26c1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:06:57.145757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 820/1646] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:06:56 2022] DELETE /identity/v3/users/31122814faeb474b84542c8cf6bd26c1 => generated 0 bytes in 407 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:06:57.150828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9730d3-1869-4f38-9861-c721e8b34917 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:57.151750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9730d3-1869-4f38-9861-c721e8b34917 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:57.152414 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b9730d3-1869-4f38-9861-c721e8b34917 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:57.209447 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6b9730d3-1869-4f38-9861-c721e8b34917 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:06:57.218167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6b9730d3-1869-4f38-9861-c721e8b34917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:57.375007 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 827/1647] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:06:57.435975 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f6214-d5db-42ee-85ca-ac4adfcda8e8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:06:57.440675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c33f6214-d5db-42ee-85ca-ac4adfcda8e8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:06:57.625417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33f6214-d5db-42ee-85ca-ac4adfcda8e8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:06:57.630967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec88a4-30a8-4e5c-a6f2-1e0e78a233fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:06:57.631795 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec88a4-30a8-4e5c-a6f2-1e0e78a233fe service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:06:57.632189 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec88a4-30a8-4e5c-a6f2-1e0e78a233fe service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:06:57.632833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcec88a4-30a8-4e5c-a6f2-1e0e78a233fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:06:57.651729 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec88a4-30a8-4e5c-a6f2-1e0e78a233fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:06:57.656928 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcec88a4-30a8-4e5c-a6f2-1e0e78a233fe service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:06:57.683197 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 821/1648] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:06:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 247 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:00.392995 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9730d3-1869-4f38-9861-c721e8b34917 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:00.394786 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6b9730d3-1869-4f38-9861-c721e8b34917 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:00.424117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b9730d3-1869-4f38-9861-c721e8b34917 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:00.443880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98d7912-2719-4e34-ba4d-d8ff358557b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:00.444711 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d7912-2719-4e34-ba4d-d8ff358557b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:00.445020 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d7912-2719-4e34-ba4d-d8ff358557b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:00.448639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98d7912-2719-4e34-ba4d-d8ff358557b2 admin admin] PATH_INFO: `/v3/projects/2c46048d0ca34522984ed0eeb26e8189` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:00.456556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98d7912-2719-4e34-ba4d-d8ff358557b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c46048d0ca34522984ed0eeb26e8189)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:00.458731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98d7912-2719-4e34-ba4d-d8ff358557b2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:00.555022 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:00.606096 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e98d7912-2719-4e34-ba4d-d8ff358557b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c46048d0ca34522984ed0eeb26e8189', 'request_id': 'req-e98d7912-2719-4e34-ba4d-d8ff358557b2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:00.647616 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e98d7912-2719-4e34-ba4d-d8ff358557b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c46048d0ca34522984ed0eeb26e8189', 'request_id': 'req-e98d7912-2719-4e34-ba4d-d8ff358557b2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:00.673820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e98d7912-2719-4e34-ba4d-d8ff358557b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c46048d0ca34522984ed0eeb26e8189', 'request_id': 'req-e98d7912-2719-4e34-ba4d-d8ff358557b2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:00.713925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e98d7912-2719-4e34-ba4d-d8ff358557b2 admin admin] The token cache is being invalidate because project 2c46048d0ca34522984ed0eeb26e8189 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:00.715000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e98d7912-2719-4e34-ba4d-d8ff358557b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:00.717128 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e98d7912-2719-4e34-ba4d-d8ff358557b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:00.725346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 828/1649] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:00 2022] DELETE /identity/v3/projects/2c46048d0ca34522984ed0eeb26e8189 => generated 0 bytes in 330 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:00.726212 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec88a4-30a8-4e5c-a6f2-1e0e78a233fe service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:00.744879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcec88a4-30a8-4e5c-a6f2-1e0e78a233fe service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:01.026777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f868bc5-cf6b-4177-8e3f-12e54a2cd20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:01.035905 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f868bc5-cf6b-4177-8e3f-12e54a2cd20e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:01.036453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f868bc5-cf6b-4177-8e3f-12e54a2cd20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:01.037030 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f868bc5-cf6b-4177-8e3f-12e54a2cd20e admin admin] PATH_INFO: `/v3/users/df4f904f36034cdb9bc4e7b28b6036b7` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:01.039543 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f868bc5-cf6b-4177-8e3f-12e54a2cd20e admin admin] RBAC: Authorizing `identity:delete_user(user_id=df4f904f36034cdb9bc4e7b28b6036b7)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:01.057798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f868bc5-cf6b-4177-8e3f-12e54a2cd20e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:01.273227 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4f868bc5-cf6b-4177-8e3f-12e54a2cd20e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'df4f904f36034cdb9bc4e7b28b6036b7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:01.274945 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=13,keystone:SELECT=86,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:01.305401 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4f868bc5-cf6b-4177-8e3f-12e54a2cd20e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'df4f904f36034cdb9bc4e7b28b6036b7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:01.324135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4f868bc5-cf6b-4177-8e3f-12e54a2cd20e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'df4f904f36034cdb9bc4e7b28b6036b7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:01.325075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4f868bc5-cf6b-4177-8e3f-12e54a2cd20e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'df4f904f36034cdb9bc4e7b28b6036b7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:01.351453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4f868bc5-cf6b-4177-8e3f-12e54a2cd20e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'df4f904f36034cdb9bc4e7b28b6036b7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:01.366267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 822/1650] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:00 2022] DELETE /identity/v3/users/df4f904f36034cdb9bc4e7b28b6036b7 => generated 0 bytes in 641 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:01.621377 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e40f67-955a-4405-89b6-6a09f12a18c7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:01.622429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e40f67-955a-4405-89b6-6a09f12a18c7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:01.623006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e40f67-955a-4405-89b6-6a09f12a18c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:01.687341 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b6e40f67-955a-4405-89b6-6a09f12a18c7 None None] MFA Rules not processed for user `04e86369906d4633b02a5ca1bcc24b35`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:01.691173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b6e40f67-955a-4405-89b6-6a09f12a18c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:01.747792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4198d7-68be-4203-b783-81fcd6ba26a1 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:01.749231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4198d7-68be-4203-b783-81fcd6ba26a1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:01.749762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4198d7-68be-4203-b783-81fcd6ba26a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:01.839399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9a4198d7-68be-4203-b783-81fcd6ba26a1 None None] MFA Rules not processed for user `cf6cc756845148ee8fce9021425cb7cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:01.840662 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9a4198d7-68be-4203-b783-81fcd6ba26a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:01.921663 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 829/1651] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:01 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 301 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:01.938279 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e40f67-955a-4405-89b6-6a09f12a18c7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:01.940244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b6e40f67-955a-4405-89b6-6a09f12a18c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:02.033436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 823/1652] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:01 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 287 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:02.043999 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29853638-32c2-445d-b25c-8571aa89a53e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:02.044286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29853638-32c2-445d-b25c-8571aa89a53e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:02.044557 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29853638-32c2-445d-b25c-8571aa89a53e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:02.093795 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-29853638-32c2-445d-b25c-8571aa89a53e None None] MFA Rules not processed for user `cf6cc756845148ee8fce9021425cb7cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:02.095589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-29853638-32c2-445d-b25c-8571aa89a53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:02.122335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 824/1653] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:02 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:02.133985 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29853638-32c2-445d-b25c-8571aa89a53e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:02.135764 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-29853638-32c2-445d-b25c-8571aa89a53e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:02.158511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e40f67-955a-4405-89b6-6a09f12a18c7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:02.163449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29853638-32c2-445d-b25c-8571aa89a53e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:02.171193 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fadecb-661f-46ba-8cb3-459900d68729 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:02.172794 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fadecb-661f-46ba-8cb3-459900d68729 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:02.173349 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fadecb-661f-46ba-8cb3-459900d68729 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:02.176707 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fadecb-661f-46ba-8cb3-459900d68729 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:02.180721 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59e013-5649-42b6-9631-ae56b5845b68 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:02.181546 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59e013-5649-42b6-9631-ae56b5845b68 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:02.181938 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59e013-5649-42b6-9631-ae56b5845b68 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:02.182351 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a59e013-5649-42b6-9631-ae56b5845b68 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:02.199448 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fadecb-661f-46ba-8cb3-459900d68729 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:02.201256 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fadecb-661f-46ba-8cb3-459900d68729 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:02.201393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59e013-5649-42b6-9631-ae56b5845b68 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:02.203128 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a59e013-5649-42b6-9631-ae56b5845b68 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:02.227782 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 825/1654] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:02.238249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 830/1655] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:01 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:07:02.980567 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59e013-5649-42b6-9631-ae56b5845b68 service nova] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:02.982639 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9a59e013-5649-42b6-9631-ae56b5845b68 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:03.133739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a59e013-5649-42b6-9631-ae56b5845b68 service nova] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:03.140427 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968853f0-b1d0-427e-87f8-dbda3a6fe1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:03.141502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968853f0-b1d0-427e-87f8-dbda3a6fe1e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:03.142801 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968853f0-b1d0-427e-87f8-dbda3a6fe1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:03.143387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968853f0-b1d0-427e-87f8-dbda3a6fe1e4 admin admin] PATH_INFO: `/v3/projects/5d9bfa59e0e840f1bebaeb5042d54a01` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:03.145167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968853f0-b1d0-427e-87f8-dbda3a6fe1e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5d9bfa59e0e840f1bebaeb5042d54a01)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:03.146705 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: WARNING py.warnings [None req-968853f0-b1d0-427e-87f8-dbda3a6fe1e4 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:07:03.146705 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: warnings.warn(msg) Jul 11 14:07:03.146705 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: Jul 11 14:07:03.148334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968853f0-b1d0-427e-87f8-dbda3a6fe1e4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:03.207613 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:03.228237 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-968853f0-b1d0-427e-87f8-dbda3a6fe1e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5d9bfa59e0e840f1bebaeb5042d54a01', 'request_id': 'req-968853f0-b1d0-427e-87f8-dbda3a6fe1e4'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:03.244139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-968853f0-b1d0-427e-87f8-dbda3a6fe1e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5d9bfa59e0e840f1bebaeb5042d54a01', 'request_id': 'req-968853f0-b1d0-427e-87f8-dbda3a6fe1e4'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:03.255135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-968853f0-b1d0-427e-87f8-dbda3a6fe1e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5d9bfa59e0e840f1bebaeb5042d54a01', 'request_id': 'req-968853f0-b1d0-427e-87f8-dbda3a6fe1e4'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:03.290878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-968853f0-b1d0-427e-87f8-dbda3a6fe1e4 admin admin] The token cache is being invalidate because project 5d9bfa59e0e840f1bebaeb5042d54a01 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:03.291441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-968853f0-b1d0-427e-87f8-dbda3a6fe1e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:03.292636 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-968853f0-b1d0-427e-87f8-dbda3a6fe1e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:03.295439 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 826/1656] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:02 2022] DELETE /identity/v3/projects/5d9bfa59e0e840f1bebaeb5042d54a01 => generated 0 bytes in 315 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:03.300178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712760b5-4836-4c3b-88dc-a4d62ef483f8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:03.301030 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712760b5-4836-4c3b-88dc-a4d62ef483f8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:03.301311 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712760b5-4836-4c3b-88dc-a4d62ef483f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:03.348086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-712760b5-4836-4c3b-88dc-a4d62ef483f8 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:03.351254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-712760b5-4836-4c3b-88dc-a4d62ef483f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:03.421115 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:03.553297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 831/1657] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 253 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:03.561164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc866fa-f64a-4ed2-9517-4fff13f8a66e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:03.561164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc866fa-f64a-4ed2-9517-4fff13f8a66e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:03.561164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fc866fa-f64a-4ed2-9517-4fff13f8a66e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:03.606746 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-0fc866fa-f64a-4ed2-9517-4fff13f8a66e None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:03.610473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0fc866fa-f64a-4ed2-9517-4fff13f8a66e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:03.638144 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 827/1658] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:03.641632 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712760b5-4836-4c3b-88dc-a4d62ef483f8 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:03.658779 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712760b5-4836-4c3b-88dc-a4d62ef483f8 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:03.671177 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5836805-859f-4e35-af6e-48dccc2b885f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:03.673052 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5836805-859f-4e35-af6e-48dccc2b885f admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:03.674930 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5836805-859f-4e35-af6e-48dccc2b885f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:03.683531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f8582-b8bc-4819-8f62-d0cfeda476aa None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:03.684198 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f8582-b8bc-4819-8f62-d0cfeda476aa None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:03.684675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-926f8582-b8bc-4819-8f62-d0cfeda476aa None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:03.694341 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 828/1659] 10.222.0.49 () {54 vars in 912 bytes} [Mon Jul 11 14:07:03 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:07:03.700705 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf0cac-624a-4cb3-a58d-985604a1f79c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:03.702619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf0cac-624a-4cb3-a58d-985604a1f79c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:03.703113 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cf0cac-624a-4cb3-a58d-985604a1f79c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:03.710643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5836805-859f-4e35-af6e-48dccc2b885f admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:03.712868 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5836805-859f-4e35-af6e-48dccc2b885f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:07:03.717281 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5836805-859f-4e35-af6e-48dccc2b885f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:03.724932 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5836805-859f-4e35-af6e-48dccc2b885f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:03.799676 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 832/1660] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:07:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:07:03.803562 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5836805-859f-4e35-af6e-48dccc2b885f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:03.832177 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5836805-859f-4e35-af6e-48dccc2b885f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:03.856478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c527833-c7ea-4c35-bf22-3ecd7a6ad78d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:03.857291 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c527833-c7ea-4c35-bf22-3ecd7a6ad78d admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:03.858951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c527833-c7ea-4c35-bf22-3ecd7a6ad78d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:03.860972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c527833-c7ea-4c35-bf22-3ecd7a6ad78d admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:03.861762 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c527833-c7ea-4c35-bf22-3ecd7a6ad78d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:03.863776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c527833-c7ea-4c35-bf22-3ecd7a6ad78d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:03.867661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-09cf0cac-624a-4cb3-a58d-985604a1f79c None None] MFA Rules not processed for user `ce1c85ee44ef45d2b631e51112b5ea0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:03.871274 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-09cf0cac-624a-4cb3-a58d-985604a1f79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:03.984563 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 833/1661] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:03 2022] POST /identity/v3/projects => generated 385 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:03.989644 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c527833-c7ea-4c35-bf22-3ecd7a6ad78d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:04.087463 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 829/1662] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 11 14:07:03 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:04.104453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf0cac-624a-4cb3-a58d-985604a1f79c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:04.106943 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-09cf0cac-624a-4cb3-a58d-985604a1f79c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:04.128838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cf0cac-624a-4cb3-a58d-985604a1f79c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:04.236733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c527833-c7ea-4c35-bf22-3ecd7a6ad78d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:04.247073 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa11956-5fd2-4c14-a291-c6786b9cdd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:04.247869 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa11956-5fd2-4c14-a291-c6786b9cdd90 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:04.248162 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa11956-5fd2-4c14-a291-c6786b9cdd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:04.251761 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa11956-5fd2-4c14-a291-c6786b9cdd90 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:04.252510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa11956-5fd2-4c14-a291-c6786b9cdd90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:04.256394 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa11956-5fd2-4c14-a291-c6786b9cdd90 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:04.284598 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bb2c8-25e8-43c0-8901-375743691997 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:04.285380 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bb2c8-25e8-43c0-8901-375743691997 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:04.285669 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bb2c8-25e8-43c0-8901-375743691997 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:04.285974 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7bb2c8-25e8-43c0-8901-375743691997 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:04.309910 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7bb2c8-25e8-43c0-8901-375743691997 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:04.311527 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7bb2c8-25e8-43c0-8901-375743691997 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:04.319573 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 834/1663] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:03 2022] POST /identity/v3/users => generated 293 bytes in 328 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:04.320711 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa11956-5fd2-4c14-a291-c6786b9cdd90 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:04.326103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 830/1664] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:07:04.343256 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa11956-5fd2-4c14-a291-c6786b9cdd90 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:04.358905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b055d9-e9a5-412f-984a-7b894f744d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:04.360004 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b055d9-e9a5-412f-984a-7b894f744d4b admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:04.360941 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b055d9-e9a5-412f-984a-7b894f744d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:04.361235 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b055d9-e9a5-412f-984a-7b894f744d4b admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:04.363437 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b055d9-e9a5-412f-984a-7b894f744d4b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:04.368265 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b055d9-e9a5-412f-984a-7b894f744d4b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:04.395033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 835/1665] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 72 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:04.396112 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bb2c8-25e8-43c0-8901-375743691997 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:04.433329 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7bb2c8-25e8-43c0-8901-375743691997 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:04.449759 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20adc86-3c43-4b84-be1d-03e89b901d7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:04.450724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20adc86-3c43-4b84-be1d-03e89b901d7d admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:04.451157 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20adc86-3c43-4b84-be1d-03e89b901d7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:04.451617 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20adc86-3c43-4b84-be1d-03e89b901d7d admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:04.452649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20adc86-3c43-4b84-be1d-03e89b901d7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:04.459549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20adc86-3c43-4b84-be1d-03e89b901d7d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:04.507486 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 831/1666] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:04 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:07:04.511965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b055d9-e9a5-412f-984a-7b894f744d4b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:04.544732 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b055d9-e9a5-412f-984a-7b894f744d4b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:04.557426 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bca0fd-ebb9-4eb7-a8f3-7c535cf1ed39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:04.559179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bca0fd-ebb9-4eb7-a8f3-7c535cf1ed39 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:04.559885 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bca0fd-ebb9-4eb7-a8f3-7c535cf1ed39 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:04.560456 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bca0fd-ebb9-4eb7-a8f3-7c535cf1ed39 admin admin] PATH_INFO: `/v3/projects/ac60d6c44bda4b7cb2fbfe80ebb873d1/users/09ab39f9e6b64b35ab050d3e4380b80f/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:04.589810 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bca0fd-ebb9-4eb7-a8f3-7c535cf1ed39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac60d6c44bda4b7cb2fbfe80ebb873d1, user_id=09ab39f9e6b64b35ab050d3e4380b80f, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:04.593925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bca0fd-ebb9-4eb7-a8f3-7c535cf1ed39 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:04.628967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 836/1667] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:04 2022] PUT /identity/v3/projects/ac60d6c44bda4b7cb2fbfe80ebb873d1/users/09ab39f9e6b64b35ab050d3e4380b80f/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:04.641437 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870d6e5-ef47-440f-a1ba-ac974a4f35c8 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:04.642361 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870d6e5-ef47-440f-a1ba-ac974a4f35c8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:04.642717 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4870d6e5-ef47-440f-a1ba-ac974a4f35c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:04.680771 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4870d6e5-ef47-440f-a1ba-ac974a4f35c8 None None] MFA Rules not processed for user `09ab39f9e6b64b35ab050d3e4380b80f`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:04.682153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4870d6e5-ef47-440f-a1ba-ac974a4f35c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:04.728334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:04.831499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bca0fd-ebb9-4eb7-a8f3-7c535cf1ed39 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:04.836441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 832/1668] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:04.976055 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:05.021693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bca0fd-ebb9-4eb7-a8f3-7c535cf1ed39 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:05.039036 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-410087f2-33f1-443d-975a-759e627b4813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:05.039996 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410087f2-33f1-443d-975a-759e627b4813 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:05.040755 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410087f2-33f1-443d-975a-759e627b4813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:05.043541 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-410087f2-33f1-443d-975a-759e627b4813 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:05.044316 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410087f2-33f1-443d-975a-759e627b4813 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:05.056510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-410087f2-33f1-443d-975a-759e627b4813 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:05.130034 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 837/1669] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:04 2022] POST /identity/v3/projects => generated 385 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:05.136521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870d6e5-ef47-440f-a1ba-ac974a4f35c8 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:05.209399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4870d6e5-ef47-440f-a1ba-ac974a4f35c8 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:05.234493 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d25911-f71a-478c-9e62-312865fe5a10 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:05.234973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d25911-f71a-478c-9e62-312865fe5a10 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:05.235370 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91d25911-f71a-478c-9e62-312865fe5a10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:05.331501 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-91d25911-f71a-478c-9e62-312865fe5a10 None None] MFA Rules not processed for user `296853a88430438e96e0f8fb97b46d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:05.332673 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-91d25911-f71a-478c-9e62-312865fe5a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:05.400501 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe6187-3398-46bf-977f-25430f8ce490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:05.401178 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe6187-3398-46bf-977f-25430f8ce490 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:05.401462 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe6187-3398-46bf-977f-25430f8ce490 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:05.401741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fbe6187-3398-46bf-977f-25430f8ce490 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:05.402479 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe6187-3398-46bf-977f-25430f8ce490 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:05.404447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fbe6187-3398-46bf-977f-25430f8ce490 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:05.473570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 833/1670] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:05 2022] POST /identity/v3/users => generated 293 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:05.479500 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe6187-3398-46bf-977f-25430f8ce490 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:05.524395 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fbe6187-3398-46bf-977f-25430f8ce490 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:05.550467 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a9f83-321d-4f8c-9961-8eaa1b1c8994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:05.551169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a9f83-321d-4f8c-9961-8eaa1b1c8994 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:05.551491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a9f83-321d-4f8c-9961-8eaa1b1c8994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:05.551719 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c70a9f83-321d-4f8c-9961-8eaa1b1c8994 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:05.552718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a9f83-321d-4f8c-9961-8eaa1b1c8994 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:05.554363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c70a9f83-321d-4f8c-9961-8eaa1b1c8994 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:05.564637 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 838/1671] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:05 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 331 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:05.590129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d25911-f71a-478c-9e62-312865fe5a10 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:05.597153 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-91d25911-f71a-478c-9e62-312865fe5a10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:05.611385 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 834/1672] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:07:05.616894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a9f83-321d-4f8c-9961-8eaa1b1c8994 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:05.654794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c70a9f83-321d-4f8c-9961-8eaa1b1c8994 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:05.684243 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-632b5374-fee6-4681-86ce-395d5ba026dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:05.684978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b5374-fee6-4681-86ce-395d5ba026dc admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:05.685268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b5374-fee6-4681-86ce-395d5ba026dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:05.685550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-632b5374-fee6-4681-86ce-395d5ba026dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:05.686296 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b5374-fee6-4681-86ce-395d5ba026dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:05.699433 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-632b5374-fee6-4681-86ce-395d5ba026dc admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:05.750788 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 835/1673] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:05 2022] GET /identity/v3/roles => generated 1420 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:05.752216 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a526c-cd07-45d1-af15-f54914766247 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:05.752506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a526c-cd07-45d1-af15-f54914766247 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:05.752788 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4a526c-cd07-45d1-af15-f54914766247 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:05.796023 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 836/1674] 10.222.0.49 () {54 vars in 960 bytes} [Mon Jul 11 14:07:05 2022] GET /identity => generated 270 bytes in 45 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:07:05.799290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a526c-cd07-45d1-af15-f54914766247 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:05.850613 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4a526c-cd07-45d1-af15-f54914766247 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:05.872724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91315c70-d67c-4aee-a813-df3c42f4b8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:05.873859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91315c70-d67c-4aee-a813-df3c42f4b8d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:05.874387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91315c70-d67c-4aee-a813-df3c42f4b8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:05.874510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91315c70-d67c-4aee-a813-df3c42f4b8d4 admin admin] PATH_INFO: `/v3/projects/6355bd126c224f3b9266dcd421fe0da3/users/f8f871a35d44447289969e4afee7f9fb/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:05.929387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91d25911-f71a-478c-9e62-312865fe5a10 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:05.937516 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91315c70-d67c-4aee-a813-df3c42f4b8d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6355bd126c224f3b9266dcd421fe0da3, user_id=f8f871a35d44447289969e4afee7f9fb, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:05.942024 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91315c70-d67c-4aee-a813-df3c42f4b8d4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:05.973512 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c41a3-122f-4371-93ca-1d20fae9467f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:05.974494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c41a3-122f-4371-93ca-1d20fae9467f service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:05.984040 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c41a3-122f-4371-93ca-1d20fae9467f service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:05.984414 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c41a3-122f-4371-93ca-1d20fae9467f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:06.012073 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 837/1675] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:05 2022] PUT /identity/v3/projects/6355bd126c224f3b9266dcd421fe0da3/users/f8f871a35d44447289969e4afee7f9fb/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 213 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:06.022027 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d93e0-7cef-437b-af44-78c1a65e6c4a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:06.022027 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d93e0-7cef-437b-af44-78c1a65e6c4a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:06.022389 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389d93e0-7cef-437b-af44-78c1a65e6c4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:06.035513 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c41a3-122f-4371-93ca-1d20fae9467f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:06.037140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c41a3-122f-4371-93ca-1d20fae9467f service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:06.095278 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 839/1676] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:05 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 515 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:06.096844 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ac5b4-153c-45eb-a70a-4b69f1182172 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:06.097132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ac5b4-153c-45eb-a70a-4b69f1182172 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:06.097413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e38ac5b4-153c-45eb-a70a-4b69f1182172 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:06.129715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-389d93e0-7cef-437b-af44-78c1a65e6c4a None None] MFA Rules not processed for user `cf6cc756845148ee8fce9021425cb7cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:06.132386 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-389d93e0-7cef-437b-af44-78c1a65e6c4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:06.200252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e38ac5b4-153c-45eb-a70a-4b69f1182172 None None] MFA Rules not processed for user `ee29b7f1cea64d6781e1df5c8021363f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:06.201493 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e38ac5b4-153c-45eb-a70a-4b69f1182172 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:06.411242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 838/1677] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:06 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 398 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:06.413019 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d90df-244f-4f08-a23f-dea8c44156cb None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:06.416182 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d90df-244f-4f08-a23f-dea8c44156cb None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:06.418600 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-884d90df-244f-4f08-a23f-dea8c44156cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:06.428669 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:06.486370 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 840/1678] 10.222.0.49 () {58 vars in 1057 bytes} [Mon Jul 11 14:07:06 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 390 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:06.493403 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-884d90df-244f-4f08-a23f-dea8c44156cb None None] MFA Rules not processed for user `f8f871a35d44447289969e4afee7f9fb`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:06.494795 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-884d90df-244f-4f08-a23f-dea8c44156cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:06.512041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ac5b4-153c-45eb-a70a-4b69f1182172 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:06.527236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e38ac5b4-153c-45eb-a70a-4b69f1182172 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:06.544872 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:06.561620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c065e8f-c069-4f66-813e-5e18f6238921 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee29b7f1cea64d6781e1df5c8021363f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:06.562506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c065e8f-c069-4f66-813e-5e18f6238921 service placement] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:06.563072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c065e8f-c069-4f66-813e-5e18f6238921 service placement] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:06.570820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c065e8f-c069-4f66-813e-5e18f6238921 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:06.583687 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7c065e8f-c069-4f66-813e-5e18f6238921 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:06.624359 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c065e8f-c069-4f66-813e-5e18f6238921 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:06.631625 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c065e8f-c069-4f66-813e-5e18f6238921 service placement] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:06.676195 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 841/1679] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:07:06.798456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 839/1680] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:06 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 383 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:06.800563 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c065e8f-c069-4f66-813e-5e18f6238921 service placement] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:06.959679 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c065e8f-c069-4f66-813e-5e18f6238921 service placement] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:06.965935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a29da7-f808-4b27-93be-fa58cc9479bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:06.966689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a29da7-f808-4b27-93be-fa58cc9479bc admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:06.966973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a29da7-f808-4b27-93be-fa58cc9479bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:06.967244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a29da7-f808-4b27-93be-fa58cc9479bc admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:06.967948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a29da7-f808-4b27-93be-fa58cc9479bc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:06.970465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a29da7-f808-4b27-93be-fa58cc9479bc admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:07.064692 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 842/1681] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:06 2022] POST /identity/v3/projects => generated 387 bytes in 265 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:07.071271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d90df-244f-4f08-a23f-dea8c44156cb None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:07.100875 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-884d90df-244f-4f08-a23f-dea8c44156cb None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:07.303945 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c5b99-ebc0-4ba9-a4da-1f0787579459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:07.304854 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c5b99-ebc0-4ba9-a4da-1f0787579459 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:07.305308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c5b99-ebc0-4ba9-a4da-1f0787579459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:07.305732 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-518c5b99-ebc0-4ba9-a4da-1f0787579459 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:07.308089 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c5b99-ebc0-4ba9-a4da-1f0787579459 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:07.313125 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-518c5b99-ebc0-4ba9-a4da-1f0787579459 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:07.362068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 840/1682] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:07 2022] POST /identity/v3/users => generated 300 bytes in 291 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:07.366010 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a29da7-f808-4b27-93be-fa58cc9479bc admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:07.377793 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a29da7-f808-4b27-93be-fa58cc9479bc admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:07.414399 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d406b88-3285-407c-b49b-f20114e62cb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:07.415380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d406b88-3285-407c-b49b-f20114e62cb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:07.415820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d406b88-3285-407c-b49b-f20114e62cb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:07.416391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d406b88-3285-407c-b49b-f20114e62cb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:07.417825 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d406b88-3285-407c-b49b-f20114e62cb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:07.424118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d406b88-3285-407c-b49b-f20114e62cb8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:07.457132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 843/1683] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:07:07.462961 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c5b99-ebc0-4ba9-a4da-1f0787579459 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:07.481360 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-518c5b99-ebc0-4ba9-a4da-1f0787579459 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:07.491490 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a407e7-e8c0-4fa0-8872-6aa8fff1dbd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:07.492762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a407e7-e8c0-4fa0-8872-6aa8fff1dbd6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:07.493050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a407e7-e8c0-4fa0-8872-6aa8fff1dbd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:07.493330 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5a407e7-e8c0-4fa0-8872-6aa8fff1dbd6 admin admin] PATH_INFO: `/v3/projects/85555e55da524204abeb8d782ddee1a9/users/b13659d602d44f0a986ed8c6acf94ef1/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:07.533121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a407e7-e8c0-4fa0-8872-6aa8fff1dbd6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=85555e55da524204abeb8d782ddee1a9, user_id=b13659d602d44f0a986ed8c6acf94ef1, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:07.536105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5a407e7-e8c0-4fa0-8872-6aa8fff1dbd6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:07.586740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 841/1684] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:07 2022] PUT /identity/v3/projects/85555e55da524204abeb8d782ddee1a9/users/b13659d602d44f0a986ed8c6acf94ef1/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:07.589803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d406b88-3285-407c-b49b-f20114e62cb8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:07.601671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d406b88-3285-407c-b49b-f20114e62cb8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:07.681778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a407e7-e8c0-4fa0-8872-6aa8fff1dbd6 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:07.685288 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d5a407e7-e8c0-4fa0-8872-6aa8fff1dbd6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:07.804880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be8f8a91-7611-4f9c-821f-96a0a790a80e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:07.806105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f8a91-7611-4f9c-821f-96a0a790a80e admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:07.807230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f8a91-7611-4f9c-821f-96a0a790a80e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:07.807815 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be8f8a91-7611-4f9c-821f-96a0a790a80e admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:07.808734 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f8a91-7611-4f9c-821f-96a0a790a80e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:07.811432 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be8f8a91-7611-4f9c-821f-96a0a790a80e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:07.833639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 844/1685] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:07 2022] GET /identity/v3/roles => generated 1420 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:07.841033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1c98c-f588-4550-a901-5940f3930e69 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:07.844999 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1c98c-f588-4550-a901-5940f3930e69 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:07.845435 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbc1c98c-f588-4550-a901-5940f3930e69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:07.855850 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5a407e7-e8c0-4fa0-8872-6aa8fff1dbd6 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:07.881900 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa348d-5b48-4f75-99f3-926e92094c79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:07.883117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa348d-5b48-4f75-99f3-926e92094c79 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:07.883407 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa348d-5b48-4f75-99f3-926e92094c79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:07.883685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fa348d-5b48-4f75-99f3-926e92094c79 admin admin] PATH_INFO: `/v3/users/85a11b01e8dd4f1e9b83c9965853a99b` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:07.886415 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa348d-5b48-4f75-99f3-926e92094c79 admin admin] RBAC: Authorizing `identity:delete_user(user_id=85a11b01e8dd4f1e9b83c9965853a99b)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:07.888578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fa348d-5b48-4f75-99f3-926e92094c79 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:07.990077 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-fbc1c98c-f588-4550-a901-5940f3930e69 None None] MFA Rules not processed for user `98143abd2e3c47e3921184a1c5e05a89`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:07.995005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fbc1c98c-f588-4550-a901-5940f3930e69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:08.010452 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=93,keystone:INSERT=4,keystone:DELETE=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:08.072701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-26fa348d-5b48-4f75-99f3-926e92094c79 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '85a11b01e8dd4f1e9b83c9965853a99b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:08.095079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:08.104945 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-26fa348d-5b48-4f75-99f3-926e92094c79 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '85a11b01e8dd4f1e9b83c9965853a99b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:08.117345 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-26fa348d-5b48-4f75-99f3-926e92094c79 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '85a11b01e8dd4f1e9b83c9965853a99b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:08.118361 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-26fa348d-5b48-4f75-99f3-926e92094c79 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '85a11b01e8dd4f1e9b83c9965853a99b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:08.137940 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-26fa348d-5b48-4f75-99f3-926e92094c79 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '85a11b01e8dd4f1e9b83c9965853a99b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:08.162954 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 842/1686] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:07 2022] DELETE /identity/v3/users/85a11b01e8dd4f1e9b83c9965853a99b => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:08.164077 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa348d-5b48-4f75-99f3-926e92094c79 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:08.166100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-26fa348d-5b48-4f75-99f3-926e92094c79 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:08.232900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 845/1687] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:08.243814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd912fc-c0f2-4a94-a909-f4c3ff685eac None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:08.244112 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd912fc-c0f2-4a94-a909-f4c3ff685eac None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:08.244401 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dd912fc-c0f2-4a94-a909-f4c3ff685eac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:08.281586 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fa348d-5b48-4f75-99f3-926e92094c79 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:08.290135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa25ac-d350-4f25-91c6-e4d766a34efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:08.291295 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa25ac-d350-4f25-91c6-e4d766a34efe admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:08.291739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa25ac-d350-4f25-91c6-e4d766a34efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:08.292165 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60fa25ac-d350-4f25-91c6-e4d766a34efe admin admin] PATH_INFO: `/v3/projects/85555e55da524204abeb8d782ddee1a9/users/b13659d602d44f0a986ed8c6acf94ef1/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:08.294718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa25ac-d350-4f25-91c6-e4d766a34efe admin admin] RBAC: Authorizing `identity:create_grant(project_id=85555e55da524204abeb8d782ddee1a9, user_id=b13659d602d44f0a986ed8c6acf94ef1, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:08.299144 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60fa25ac-d350-4f25-91c6-e4d766a34efe admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:08.307494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6dd912fc-c0f2-4a94-a909-f4c3ff685eac None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:08.308874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6dd912fc-c0f2-4a94-a909-f4c3ff685eac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:08.324268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 843/1688] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:08 2022] PUT /identity/v3/projects/85555e55da524204abeb8d782ddee1a9/users/b13659d602d44f0a986ed8c6acf94ef1/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:08.325125 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa25ac-d350-4f25-91c6-e4d766a34efe admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:08.331640 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-60fa25ac-d350-4f25-91c6-e4d766a34efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:08.360319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 846/1689] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:08.361784 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6dc06-bd31-4554-ab5d-138f8fec746e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:08.362055 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6dc06-bd31-4554-ab5d-138f8fec746e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:08.362370 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fde6dc06-bd31-4554-ab5d-138f8fec746e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:08.451932 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-fde6dc06-bd31-4554-ab5d-138f8fec746e None None] MFA Rules not processed for user `b13659d602d44f0a986ed8c6acf94ef1`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:08.453167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fde6dc06-bd31-4554-ab5d-138f8fec746e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:08.483479 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60fa25ac-d350-4f25-91c6-e4d766a34efe admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:08.512222 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de9b70-06ce-4acd-ad06-cac30a87278d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:08.513042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de9b70-06ce-4acd-ad06-cac30a87278d service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:08.513330 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de9b70-06ce-4acd-ad06-cac30a87278d service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:08.514078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80de9b70-06ce-4acd-ad06-cac30a87278d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:08.642060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=7,keystone:SELECT=91,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:08.752747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 847/1690] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:08 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 392 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:08.753915 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfc72c-2c15-4c53-8e44-19c2afa9b931 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:08.754205 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfc72c-2c15-4c53-8e44-19c2afa9b931 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:08.757144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2dfc72c-2c15-4c53-8e44-19c2afa9b931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:08.761791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80de9b70-06ce-4acd-ad06-cac30a87278d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:08.763389 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80de9b70-06ce-4acd-ad06-cac30a87278d service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:08.804714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b2dfc72c-2c15-4c53-8e44-19c2afa9b931 None None] MFA Rules not processed for user `170f249dccd4498990a8344a18c73059`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:08.806153 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b2dfc72c-2c15-4c53-8e44-19c2afa9b931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:08.807216 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 844/1691] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:08 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 482 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:08.808523 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de9b70-06ce-4acd-ad06-cac30a87278d service nova] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:08.817511 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-80de9b70-06ce-4acd-ad06-cac30a87278d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:08.903908 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 848/1692] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:08 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:08.904917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfc72c-2c15-4c53-8e44-19c2afa9b931 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:08.906247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b2dfc72c-2c15-4c53-8e44-19c2afa9b931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:08.984636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:09.085034 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2dfc72c-2c15-4c53-8e44-19c2afa9b931 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:09.086615 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80de9b70-06ce-4acd-ad06-cac30a87278d service nova] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:09.091383 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f194baa-e2e2-4f94-ba19-430e9b341fdf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:09.092219 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f194baa-e2e2-4f94-ba19-430e9b341fdf service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:09.092621 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f194baa-e2e2-4f94-ba19-430e9b341fdf service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:09.093018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f194baa-e2e2-4f94-ba19-430e9b341fdf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:09.102921 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b683a6-0fb4-491e-991d-8d74d37c6844 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:09.106841 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b683a6-0fb4-491e-991d-8d74d37c6844 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:09.107336 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b683a6-0fb4-491e-991d-8d74d37c6844 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:09.107766 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b683a6-0fb4-491e-991d-8d74d37c6844 admin admin] PATH_INFO: `/v3/users/c3d3348c338b4567bfea396fa7645a17` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:09.114027 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b683a6-0fb4-491e-991d-8d74d37c6844 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3d3348c338b4567bfea396fa7645a17)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:09.114644 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f194baa-e2e2-4f94-ba19-430e9b341fdf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:09.115947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: WARNING py.warnings [None req-60b683a6-0fb4-491e-991d-8d74d37c6844 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 11 14:07:09.115947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: warnings.warn(msg) Jul 11 14:07:09.115947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: Jul 11 14:07:09.118000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b683a6-0fb4-491e-991d-8d74d37c6844 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:09.118563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f194baa-e2e2-4f94-ba19-430e9b341fdf service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:09.157044 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 845/1693] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:09.158881 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f194baa-e2e2-4f94-ba19-430e9b341fdf service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:09.162544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4f194baa-e2e2-4f94-ba19-430e9b341fdf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:09.404746 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-60b683a6-0fb4-491e-991d-8d74d37c6844 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3d3348c338b4567bfea396fa7645a17'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:09.411664 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-60b683a6-0fb4-491e-991d-8d74d37c6844 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3d3348c338b4567bfea396fa7645a17'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:09.440307 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-60b683a6-0fb4-491e-991d-8d74d37c6844 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3d3348c338b4567bfea396fa7645a17'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:09.441106 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-60b683a6-0fb4-491e-991d-8d74d37c6844 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3d3348c338b4567bfea396fa7645a17'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:09.445743 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f194baa-e2e2-4f94-ba19-430e9b341fdf service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:09.466724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a8d64-c967-4ac4-aa32-c87f77e19307 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:09.467509 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a8d64-c967-4ac4-aa32-c87f77e19307 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:09.468648 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a8d64-c967-4ac4-aa32-c87f77e19307 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:09.470240 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a8d64-c967-4ac4-aa32-c87f77e19307 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:09.478210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8e2a8d64-c967-4ac4-aa32-c87f77e19307 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:09.509964 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-60b683a6-0fb4-491e-991d-8d74d37c6844 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3d3348c338b4567bfea396fa7645a17'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:09.526005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a8d64-c967-4ac4-aa32-c87f77e19307 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:09.534020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a8d64-c967-4ac4-aa32-c87f77e19307 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:09.547436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 846/1694] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:09 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:09.550622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50660798-43a4-453e-990f-15617fc3b8c2 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:09.551323 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50660798-43a4-453e-990f-15617fc3b8c2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:09.552349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50660798-43a4-453e-990f-15617fc3b8c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:09.581133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-50660798-43a4-453e-990f-15617fc3b8c2 None None] MFA Rules not processed for user `170f249dccd4498990a8344a18c73059`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:09.583590 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-50660798-43a4-453e-990f-15617fc3b8c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:09.584299 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 849/1695] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:08 2022] DELETE /identity/v3/users/c3d3348c338b4567bfea396fa7645a17 => generated 0 bytes in 680 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:09.591775 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575637d-be0e-449a-91dc-c339bedd005c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:09.592059 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575637d-be0e-449a-91dc-c339bedd005c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:09.592346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6575637d-be0e-449a-91dc-c339bedd005c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:09.655658 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6575637d-be0e-449a-91dc-c339bedd005c None None] MFA Rules not processed for user `04e86369906d4633b02a5ca1bcc24b35`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:09.656926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6575637d-be0e-449a-91dc-c339bedd005c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:09.740685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 847/1696] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:09.743531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac28177-22b3-4249-a416-998e2e50d78c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:09.744113 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac28177-22b3-4249-a416-998e2e50d78c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:09.744409 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac28177-22b3-4249-a416-998e2e50d78c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:09.784485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-6ac28177-22b3-4249-a416-998e2e50d78c None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:09.785982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6ac28177-22b3-4249-a416-998e2e50d78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:09.862591 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:09.865368 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 850/1697] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:09 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 281 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:09.869361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575637d-be0e-449a-91dc-c339bedd005c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:09.877066 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6575637d-be0e-449a-91dc-c339bedd005c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:10.096838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 848/1698] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:10.186541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac28177-22b3-4249-a416-998e2e50d78c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:10.189035 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6ac28177-22b3-4249-a416-998e2e50d78c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:10.195612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:10.214845 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6575637d-be0e-449a-91dc-c339bedd005c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:10.230720 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd31d1-d79d-47ea-9b45-1dc15fac139f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:10.231604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd31d1-d79d-47ea-9b45-1dc15fac139f service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:10.232057 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd31d1-d79d-47ea-9b45-1dc15fac139f service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:10.232480 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd31d1-d79d-47ea-9b45-1dc15fac139f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:10.236533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bffd31d1-d79d-47ea-9b45-1dc15fac139f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:10.293799 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd31d1-d79d-47ea-9b45-1dc15fac139f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:10.297072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd31d1-d79d-47ea-9b45-1dc15fac139f service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:10.365352 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 851/1699] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:09 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 499 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:10.435727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac28177-22b3-4249-a416-998e2e50d78c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:10.449270 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f7d093a-4030-4081-a734-d24f8fcbaaca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:10.449894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d093a-4030-4081-a734-d24f8fcbaaca service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:10.450197 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d093a-4030-4081-a734-d24f8fcbaaca service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:10.450501 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f7d093a-4030-4081-a734-d24f8fcbaaca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:10.492080 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d093a-4030-4081-a734-d24f8fcbaaca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:10.499976 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f7d093a-4030-4081-a734-d24f8fcbaaca service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:10.562059 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 849/1700] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:07:11.068800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd31d1-d79d-47ea-9b45-1dc15fac139f service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:11.070244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bffd31d1-d79d-47ea-9b45-1dc15fac139f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:11.099501 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe9895-0df2-40e9-bf77-2c723ed6deab None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:11.099712 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe9895-0df2-40e9-bf77-2c723ed6deab None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:11.100051 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1efe9895-0df2-40e9-bf77-2c723ed6deab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:11.119615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd31d1-d79d-47ea-9b45-1dc15fac139f service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:11.139206 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87d850-7e6a-4ca8-a9c5-f105f9e060aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:11.150642 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d850-7e6a-4ca8-a9c5-f105f9e060aa admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:11.151345 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d850-7e6a-4ca8-a9c5-f105f9e060aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:11.152404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87d850-7e6a-4ca8-a9c5-f105f9e060aa admin admin] PATH_INFO: `/v3/projects/15fd934ea11040fe93f64b06bb1e4e21` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:11.158498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87d850-7e6a-4ca8-a9c5-f105f9e060aa admin admin] RBAC: Authorizing `identity:delete_project(project_id=15fd934ea11040fe93f64b06bb1e4e21)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:11.160525 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed87d850-7e6a-4ca8-a9c5-f105f9e060aa admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:11.306326 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1efe9895-0df2-40e9-bf77-2c723ed6deab None None] MFA Rules not processed for user `52076f59f68f4d77a0dadbf7a98d2b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:11.308222 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1efe9895-0df2-40e9-bf77-2c723ed6deab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:11.333623 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ed87d850-7e6a-4ca8-a9c5-f105f9e060aa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '15fd934ea11040fe93f64b06bb1e4e21', 'request_id': 'req-ed87d850-7e6a-4ca8-a9c5-f105f9e060aa'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:11.340868 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ed87d850-7e6a-4ca8-a9c5-f105f9e060aa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '15fd934ea11040fe93f64b06bb1e4e21', 'request_id': 'req-ed87d850-7e6a-4ca8-a9c5-f105f9e060aa'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:11.360063 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ed87d850-7e6a-4ca8-a9c5-f105f9e060aa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '15fd934ea11040fe93f64b06bb1e4e21', 'request_id': 'req-ed87d850-7e6a-4ca8-a9c5-f105f9e060aa'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:11.386234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ed87d850-7e6a-4ca8-a9c5-f105f9e060aa admin admin] The token cache is being invalidate because project 15fd934ea11040fe93f64b06bb1e4e21 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:11.386861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ed87d850-7e6a-4ca8-a9c5-f105f9e060aa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:11.387816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ed87d850-7e6a-4ca8-a9c5-f105f9e060aa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:11.389945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 852/1701] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:11 2022] DELETE /identity/v3/projects/15fd934ea11040fe93f64b06bb1e4e21 => generated 0 bytes in 321 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:11.391070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87d850-7e6a-4ca8-a9c5-f105f9e060aa admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:11.394896 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ed87d850-7e6a-4ca8-a9c5-f105f9e060aa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:11.538368 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87d850-7e6a-4ca8-a9c5-f105f9e060aa admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:11.541761 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 850/1702] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 11 14:07:11 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 443 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:11.545319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d92334-4ecc-4e5e-be76-b4fdc03718a3 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:11.546606 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d92334-4ecc-4e5e-be76-b4fdc03718a3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:11.546983 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d92334-4ecc-4e5e-be76-b4fdc03718a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:11.582966 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ef5e5-d230-4c57-acc7-471f6e9df703 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:11.588452 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ef5e5-d230-4c57-acc7-471f6e9df703 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:11.589553 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ef5e5-d230-4c57-acc7-471f6e9df703 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:11.590349 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e03ef5e5-d230-4c57-acc7-471f6e9df703 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:11.598686 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-86d92334-4ecc-4e5e-be76-b4fdc03718a3 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:11.599715 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e03ef5e5-d230-4c57-acc7-471f6e9df703 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:11.599950 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-86d92334-4ecc-4e5e-be76-b4fdc03718a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:11.721832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:11.790472 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 851/1703] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 248 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:11.791402 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ef5e5-d230-4c57-acc7-471f6e9df703 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:11.792170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49851a43-6d49-4391-ada3-34a171243be8 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:11.792478 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49851a43-6d49-4391-ada3-34a171243be8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:11.792743 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49851a43-6d49-4391-ada3-34a171243be8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:11.803509 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e03ef5e5-d230-4c57-acc7-471f6e9df703 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:11.834752 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-49851a43-6d49-4391-ada3-34a171243be8 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:11.835957 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-49851a43-6d49-4391-ada3-34a171243be8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:11.845743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 853/1704] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:11 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 455 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:11.875360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ef5e5-d230-4c57-acc7-471f6e9df703 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:11.877226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e03ef5e5-d230-4c57-acc7-471f6e9df703 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:11.880706 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 852/1705] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:11.887650 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49851a43-6d49-4391-ada3-34a171243be8 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:11.913736 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49851a43-6d49-4391-ada3-34a171243be8 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:11.944180 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84aa820-f629-4c61-99cc-fff4c32cabbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:11.944870 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84aa820-f629-4c61-99cc-fff4c32cabbf admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:11.945153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84aa820-f629-4c61-99cc-fff4c32cabbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:11.945432 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84aa820-f629-4c61-99cc-fff4c32cabbf admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:11.946183 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84aa820-f629-4c61-99cc-fff4c32cabbf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:07:11.946504 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84aa820-f629-4c61-99cc-fff4c32cabbf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:11.959388 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:11.960378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84aa820-f629-4c61-99cc-fff4c32cabbf admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:11.994445 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 853/1706] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:07:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:07:11.995395 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84aa820-f629-4c61-99cc-fff4c32cabbf admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:12.033610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84aa820-f629-4c61-99cc-fff4c32cabbf admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:12.058570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348da502-124f-47cf-b3f5-786aa076b967 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:12.059606 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348da502-124f-47cf-b3f5-786aa076b967 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:12.060190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348da502-124f-47cf-b3f5-786aa076b967 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:12.060620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-348da502-124f-47cf-b3f5-786aa076b967 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:12.061465 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348da502-124f-47cf-b3f5-786aa076b967 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:12.063792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-348da502-124f-47cf-b3f5-786aa076b967 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:12.106597 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 854/1707] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:11 2022] POST /identity/v3/projects => generated 385 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:12.108060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348da502-124f-47cf-b3f5-786aa076b967 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:12.156005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e03ef5e5-d230-4c57-acc7-471f6e9df703 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:12.291766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-348da502-124f-47cf-b3f5-786aa076b967 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:12.306968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9c879-d54e-4280-b7bb-4d439b495efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:12.314572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9c879-d54e-4280-b7bb-4d439b495efb admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:12.315270 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9c879-d54e-4280-b7bb-4d439b495efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:12.316406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b9c879-d54e-4280-b7bb-4d439b495efb admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:12.317744 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b9c879-d54e-4280-b7bb-4d439b495efb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:12.320110 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66b9c879-d54e-4280-b7bb-4d439b495efb admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:12.358034 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6500ff-9922-494d-928d-447c3c881fc8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c1369033a44c1c81d4c1c9ff2bbedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:12.358958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6500ff-9922-494d-928d-447c3c881fc8 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:12.359488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6500ff-9922-494d-928d-447c3c881fc8 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:12.360362 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c6500ff-9922-494d-928d-447c3c881fc8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:12.394983 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 855/1708] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:12 2022] POST /identity/v3/users => generated 293 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:12.398866 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9c879-d54e-4280-b7bb-4d439b495efb admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:12.426272 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66b9c879-d54e-4280-b7bb-4d439b495efb admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:12.442371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30b4bc-9a8b-4a63-976e-260b34e59dbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:12.446396 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30b4bc-9a8b-4a63-976e-260b34e59dbf admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:12.446792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30b4bc-9a8b-4a63-976e-260b34e59dbf admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:12.447103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b30b4bc-9a8b-4a63-976e-260b34e59dbf admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:12.447902 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b30b4bc-9a8b-4a63-976e-260b34e59dbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:12.454583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b30b4bc-9a8b-4a63-976e-260b34e59dbf admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:12.495886 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 856/1709] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:12.500497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30b4bc-9a8b-4a63-976e-260b34e59dbf admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:12.546444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b30b4bc-9a8b-4a63-976e-260b34e59dbf admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:12.560237 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09505d11-09d5-4ec9-b66c-9babc398ea19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:12.565494 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09505d11-09d5-4ec9-b66c-9babc398ea19 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:12.566041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09505d11-09d5-4ec9-b66c-9babc398ea19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:12.566690 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6500ff-9922-494d-928d-447c3c881fc8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:12.567939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09505d11-09d5-4ec9-b66c-9babc398ea19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:12.568884 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09505d11-09d5-4ec9-b66c-9babc398ea19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:12.569083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c6500ff-9922-494d-928d-447c3c881fc8 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:12.573263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09505d11-09d5-4ec9-b66c-9babc398ea19 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:12.582190 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 854/1710] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 11 14:07:11 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 707 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 11 14:07:12.617609 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 857/1711] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:12 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:12.620579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6500ff-9922-494d-928d-447c3c881fc8 service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:12.636839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c6500ff-9922-494d-928d-447c3c881fc8 service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:12.644234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c70972-a219-4690-a284-cf5ccc971a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:12.645314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c70972-a219-4690-a284-cf5ccc971a5c admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:12.645712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c70972-a219-4690-a284-cf5ccc971a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:12.646100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c70972-a219-4690-a284-cf5ccc971a5c admin admin] PATH_INFO: `/v3/projects/036d8b8abfd044b1b53efde9ba5854b9/users/5294d35cd9894056b84395f063ba4e09/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:12.674556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c70972-a219-4690-a284-cf5ccc971a5c admin admin] RBAC: Authorizing `identity:create_grant(project_id=036d8b8abfd044b1b53efde9ba5854b9, user_id=5294d35cd9894056b84395f063ba4e09, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:12.678190 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1c70972-a219-4690-a284-cf5ccc971a5c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:12.712636 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de40a29-6b6b-45a8-b759-b44cbe68d5e7 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:12.713619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de40a29-6b6b-45a8-b759-b44cbe68d5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:12.714174 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de40a29-6b6b-45a8-b759-b44cbe68d5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:12.723220 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 855/1712] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:12 2022] PUT /identity/v3/projects/036d8b8abfd044b1b53efde9ba5854b9/users/5294d35cd9894056b84395f063ba4e09/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:12.840287 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1de40a29-6b6b-45a8-b759-b44cbe68d5e7 None None] MFA Rules not processed for user `5294d35cd9894056b84395f063ba4e09`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:12.849939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1de40a29-6b6b-45a8-b759-b44cbe68d5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:12.855571 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9d400-584a-45a9-bbd7-5e93243905ae None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:12.856082 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9d400-584a-45a9-bbd7-5e93243905ae None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:12.856598 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bb9d400-584a-45a9-bbd7-5e93243905ae None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:12.858771 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 856/1713] 10.222.0.49 () {54 vars in 912 bytes} [Mon Jul 11 14:07:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:07:12.870631 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587b295-9e30-4846-927e-1afd6f1c0b1e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:12.871089 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587b295-9e30-4846-927e-1afd6f1c0b1e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:12.871480 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9587b295-9e30-4846-927e-1afd6f1c0b1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:12.900169 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9587b295-9e30-4846-927e-1afd6f1c0b1e None None] MFA Rules not processed for user `ce1c85ee44ef45d2b631e51112b5ea0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:12.901615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9587b295-9e30-4846-927e-1afd6f1c0b1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:12.955121 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:13.040948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 857/1714] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 11 14:07:12 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:13.053262 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:13.111184 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 858/1715] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:12 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 396 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:13.112898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587b295-9e30-4846-927e-1afd6f1c0b1e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:13.351761 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9587b295-9e30-4846-927e-1afd6f1c0b1e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:13.391315 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee8e93-32b2-4f5c-b6b7-ce077e72e6c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:13.393032 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee8e93-32b2-4f5c-b6b7-ce077e72e6c9 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:13.393512 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee8e93-32b2-4f5c-b6b7-ce077e72e6c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:13.393980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ee8e93-32b2-4f5c-b6b7-ce077e72e6c9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:13.395124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee8e93-32b2-4f5c-b6b7-ce077e72e6c9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:13.406119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ee8e93-32b2-4f5c-b6b7-ce077e72e6c9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:13.472825 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 858/1716] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:13 2022] POST /identity/v3/projects => generated 385 bytes in 360 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:13.476741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de40a29-6b6b-45a8-b759-b44cbe68d5e7 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:13.533521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de40a29-6b6b-45a8-b759-b44cbe68d5e7 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:13.695400 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb2489-e4df-4e0c-8d58-4f51d5ec0f8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:13.696324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb2489-e4df-4e0c-8d58-4f51d5ec0f8c admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:13.696719 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb2489-e4df-4e0c-8d58-4f51d5ec0f8c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:13.697156 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fb2489-e4df-4e0c-8d58-4f51d5ec0f8c admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:13.698064 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb2489-e4df-4e0c-8d58-4f51d5ec0f8c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:13.703055 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fb2489-e4df-4e0c-8d58-4f51d5ec0f8c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:13.752194 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 859/1717] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:13 2022] POST /identity/v3/users => generated 299 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:13.755904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee8e93-32b2-4f5c-b6b7-ce077e72e6c9 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:13.767858 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ee8e93-32b2-4f5c-b6b7-ce077e72e6c9 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:13.781935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d823f-3e79-424f-9b63-9f971c2a971b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:13.782868 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d823f-3e79-424f-9b63-9f971c2a971b admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:13.783344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d823f-3e79-424f-9b63-9f971c2a971b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:13.783775 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-222d823f-3e79-424f-9b63-9f971c2a971b admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:13.784757 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d823f-3e79-424f-9b63-9f971c2a971b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:13.786985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-222d823f-3e79-424f-9b63-9f971c2a971b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:13.791432 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb2489-e4df-4e0c-8d58-4f51d5ec0f8c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:13.793601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d8fb2489-e4df-4e0c-8d58-4f51d5ec0f8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:13.849378 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 859/1718] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:13 2022] GET /identity/v3/roles => generated 1420 bytes in 94 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:13.854734 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d823f-3e79-424f-9b63-9f971c2a971b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:13.890927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-222d823f-3e79-424f-9b63-9f971c2a971b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:13.908364 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d66de-fe7a-48b3-8d16-d8fc1c7236dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:13.909439 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d66de-fe7a-48b3-8d16-d8fc1c7236dc admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:13.910662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d66de-fe7a-48b3-8d16-d8fc1c7236dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:13.911296 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07d66de-fe7a-48b3-8d16-d8fc1c7236dc admin admin] PATH_INFO: `/v3/projects/6f60d4a9b9b0474883a1b58607111c6d/users/cc5a9253b1624b2ebfaae540ba73f768/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:13.962445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d66de-fe7a-48b3-8d16-d8fc1c7236dc admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f60d4a9b9b0474883a1b58607111c6d, user_id=cc5a9253b1624b2ebfaae540ba73f768, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:13.965579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07d66de-fe7a-48b3-8d16-d8fc1c7236dc admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:13.992638 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fb2489-e4df-4e0c-8d58-4f51d5ec0f8c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:14.012005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 860/1719] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:13 2022] PUT /identity/v3/projects/6f60d4a9b9b0474883a1b58607111c6d/users/cc5a9253b1624b2ebfaae540ba73f768/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:14.016002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9d1a5-cf72-45f1-bf7b-cbe060f14cfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:14.017025 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9d1a5-cf72-45f1-bf7b-cbe060f14cfd service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:14.018222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d66de-fe7a-48b3-8d16-d8fc1c7236dc admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:14.018736 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9d1a5-cf72-45f1-bf7b-cbe060f14cfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:14.019487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ba9d1a5-cf72-45f1-bf7b-cbe060f14cfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:14.049016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba9d1a5-cf72-45f1-bf7b-cbe060f14cfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:14.050879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ba9d1a5-cf72-45f1-bf7b-cbe060f14cfd service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:14.084325 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 860/1720] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:07:14.238416 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9d1a5-cf72-45f1-bf7b-cbe060f14cfd service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:14.240945 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3ba9d1a5-cf72-45f1-bf7b-cbe060f14cfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:14.275455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07d66de-fe7a-48b3-8d16-d8fc1c7236dc admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:14.284392 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ba9d1a5-cf72-45f1-bf7b-cbe060f14cfd service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:14.296651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393a10d-5fa2-44f8-99d7-15ca5079eb07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:14.298481 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393a10d-5fa2-44f8-99d7-15ca5079eb07 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:14.298772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393a10d-5fa2-44f8-99d7-15ca5079eb07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:14.299049 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f393a10d-5fa2-44f8-99d7-15ca5079eb07 admin admin] PATH_INFO: `/v3/projects/3b9e53c9298348959c10d9e79a7a5e63` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:14.300726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf02b27-a2bb-4187-98d7-33f478cb79ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:14.302496 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393a10d-5fa2-44f8-99d7-15ca5079eb07 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b9e53c9298348959c10d9e79a7a5e63)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:14.304439 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf02b27-a2bb-4187-98d7-33f478cb79ad admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:14.305138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf02b27-a2bb-4187-98d7-33f478cb79ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:14.306232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cf02b27-a2bb-4187-98d7-33f478cb79ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:14.310458 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f393a10d-5fa2-44f8-99d7-15ca5079eb07 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:14.325048 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf02b27-a2bb-4187-98d7-33f478cb79ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:14.337274 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cf02b27-a2bb-4187-98d7-33f478cb79ad admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:14.401952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 861/1721] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:14 2022] GET /identity/v3/roles => generated 1420 bytes in 383 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:14.403783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf02b27-a2bb-4187-98d7-33f478cb79ad admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:14.415102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f393a10d-5fa2-44f8-99d7-15ca5079eb07 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b9e53c9298348959c10d9e79a7a5e63', 'request_id': 'req-f393a10d-5fa2-44f8-99d7-15ca5079eb07'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:14.427716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f393a10d-5fa2-44f8-99d7-15ca5079eb07 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b9e53c9298348959c10d9e79a7a5e63', 'request_id': 'req-f393a10d-5fa2-44f8-99d7-15ca5079eb07'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:14.445890 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f393a10d-5fa2-44f8-99d7-15ca5079eb07 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b9e53c9298348959c10d9e79a7a5e63', 'request_id': 'req-f393a10d-5fa2-44f8-99d7-15ca5079eb07'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:14.451841 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cf02b27-a2bb-4187-98d7-33f478cb79ad admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:14.490220 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:14.498446 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f393a10d-5fa2-44f8-99d7-15ca5079eb07 admin admin] The token cache is being invalidate because project 3b9e53c9298348959c10d9e79a7a5e63 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:14.501491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f393a10d-5fa2-44f8-99d7-15ca5079eb07 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:14.503849 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f393a10d-5fa2-44f8-99d7-15ca5079eb07 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:14.513020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 861/1722] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:14 2022] DELETE /identity/v3/projects/3b9e53c9298348959c10d9e79a7a5e63 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:14.514769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393a10d-5fa2-44f8-99d7-15ca5079eb07 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:14.518777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f393a10d-5fa2-44f8-99d7-15ca5079eb07 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:14.557376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca62f648-d392-4f28-bee7-342d0d6eec19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:14.558344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62f648-d392-4f28-bee7-342d0d6eec19 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:14.558912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62f648-d392-4f28-bee7-342d0d6eec19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:14.559286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca62f648-d392-4f28-bee7-342d0d6eec19 admin admin] PATH_INFO: `/v3/projects/6f60d4a9b9b0474883a1b58607111c6d/users/cc5a9253b1624b2ebfaae540ba73f768/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:14.562069 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62f648-d392-4f28-bee7-342d0d6eec19 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f60d4a9b9b0474883a1b58607111c6d, user_id=cc5a9253b1624b2ebfaae540ba73f768, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:14.564528 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:14.566245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca62f648-d392-4f28-bee7-342d0d6eec19 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:14.582507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 862/1723] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:14 2022] PUT /identity/v3/projects/6f60d4a9b9b0474883a1b58607111c6d/users/cc5a9253b1624b2ebfaae540ba73f768/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:14.595722 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcceb99-fd0f-4837-9339-1996cb06d2e9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:14.596531 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcceb99-fd0f-4837-9339-1996cb06d2e9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:14.597034 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebcceb99-fd0f-4837-9339-1996cb06d2e9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:14.698251 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ebcceb99-fd0f-4837-9339-1996cb06d2e9 None None] MFA Rules not processed for user `cc5a9253b1624b2ebfaae540ba73f768`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:14.705365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ebcceb99-fd0f-4837-9339-1996cb06d2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:14.789489 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f393a10d-5fa2-44f8-99d7-15ca5079eb07 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:14.959989 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 863/1724] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:14 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:15.000980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcceb99-fd0f-4837-9339-1996cb06d2e9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:15.003153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d826632-ab06-45b7-8bda-174ad93f30d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:15.003595 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d826632-ab06-45b7-8bda-174ad93f30d4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:15.003886 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d826632-ab06-45b7-8bda-174ad93f30d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:15.004166 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d826632-ab06-45b7-8bda-174ad93f30d4 admin admin] PATH_INFO: `/v3/users/76e1951a4e014d1ca1fabe64c0792285` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:15.005477 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d826632-ab06-45b7-8bda-174ad93f30d4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=76e1951a4e014d1ca1fabe64c0792285)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:15.010171 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d826632-ab06-45b7-8bda-174ad93f30d4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:15.011528 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ebcceb99-fd0f-4837-9339-1996cb06d2e9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:15.172046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3d826632-ab06-45b7-8bda-174ad93f30d4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '76e1951a4e014d1ca1fabe64c0792285'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:15.209867 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3d826632-ab06-45b7-8bda-174ad93f30d4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '76e1951a4e014d1ca1fabe64c0792285'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:15.219526 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebcceb99-fd0f-4837-9339-1996cb06d2e9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:15.225812 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d38ba-8be9-4fbe-997f-a0e589d136ce service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:15.229032 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d38ba-8be9-4fbe-997f-a0e589d136ce service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:15.229811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d38ba-8be9-4fbe-997f-a0e589d136ce service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:15.230608 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea8d38ba-8be9-4fbe-997f-a0e589d136ce service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:15.240264 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3d826632-ab06-45b7-8bda-174ad93f30d4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '76e1951a4e014d1ca1fabe64c0792285'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:15.241011 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3d826632-ab06-45b7-8bda-174ad93f30d4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '76e1951a4e014d1ca1fabe64c0792285'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:15.264237 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d38ba-8be9-4fbe-997f-a0e589d136ce service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:15.265722 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea8d38ba-8be9-4fbe-997f-a0e589d136ce service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:15.297075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 864/1725] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:14 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 301 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:15.303001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3d826632-ab06-45b7-8bda-174ad93f30d4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '76e1951a4e014d1ca1fabe64c0792285'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:15.315540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 862/1726] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:14 2022] DELETE /identity/v3/users/76e1951a4e014d1ca1fabe64c0792285 => generated 0 bytes in 802 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:15.519057 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d38ba-8be9-4fbe-997f-a0e589d136ce service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:15.521415 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ea8d38ba-8be9-4fbe-997f-a0e589d136ce service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:15.547995 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea8d38ba-8be9-4fbe-997f-a0e589d136ce service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:15.688650 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:15.732304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6349539-f38e-41db-8ae5-95e799d15c50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:15.733578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6349539-f38e-41db-8ae5-95e799d15c50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:15.734113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6349539-f38e-41db-8ae5-95e799d15c50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:15.734617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6349539-f38e-41db-8ae5-95e799d15c50 admin admin] PATH_INFO: `/v3/users/5294d35cd9894056b84395f063ba4e09` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:15.738089 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6349539-f38e-41db-8ae5-95e799d15c50 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5294d35cd9894056b84395f063ba4e09)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:15.746765 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6349539-f38e-41db-8ae5-95e799d15c50 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:15.858908 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e6349539-f38e-41db-8ae5-95e799d15c50 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5294d35cd9894056b84395f063ba4e09'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:15.861975 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e6349539-f38e-41db-8ae5-95e799d15c50 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5294d35cd9894056b84395f063ba4e09'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:15.886439 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e6349539-f38e-41db-8ae5-95e799d15c50 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5294d35cd9894056b84395f063ba4e09'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:15.891943 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e6349539-f38e-41db-8ae5-95e799d15c50 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5294d35cd9894056b84395f063ba4e09'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:15.938665 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e6349539-f38e-41db-8ae5-95e799d15c50 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5294d35cd9894056b84395f063ba4e09'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:16.029683 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 865/1727] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:15 2022] DELETE /identity/v3/users/5294d35cd9894056b84395f063ba4e09 => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:16.032000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729f3b4-d290-41a0-9bc9-30549c43e516 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:16.032499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729f3b4-d290-41a0-9bc9-30549c43e516 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:16.032988 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4729f3b4-d290-41a0-9bc9-30549c43e516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:16.098205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4729f3b4-d290-41a0-9bc9-30549c43e516 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:16.113015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4729f3b4-d290-41a0-9bc9-30549c43e516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:16.232164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:16.342421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 863/1728] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 312 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:16.442282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6349539-f38e-41db-8ae5-95e799d15c50 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:16.442282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e6349539-f38e-41db-8ae5-95e799d15c50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:16.452644 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b1475-0bf3-4182-a2db-3ae13c6f1ff4 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:16.453177 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b1475-0bf3-4182-a2db-3ae13c6f1ff4 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:16.453754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1b1475-0bf3-4182-a2db-3ae13c6f1ff4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:16.518075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8c1b1475-0bf3-4182-a2db-3ae13c6f1ff4 None None] MFA Rules not processed for user `98143abd2e3c47e3921184a1c5e05a89`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:16.525815 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8c1b1475-0bf3-4182-a2db-3ae13c6f1ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:16.641880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6349539-f38e-41db-8ae5-95e799d15c50 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:16.646060 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482201ce-16f6-4c1c-b89d-d287a9dc16ae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:16.646950 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482201ce-16f6-4c1c-b89d-d287a9dc16ae service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:16.647456 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482201ce-16f6-4c1c-b89d-d287a9dc16ae service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:16.647842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482201ce-16f6-4c1c-b89d-d287a9dc16ae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:16.659474 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482201ce-16f6-4c1c-b89d-d287a9dc16ae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:16.661551 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482201ce-16f6-4c1c-b89d-d287a9dc16ae service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:16.677379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 866/1729] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 240 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:16.784079 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 864/1730] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:16 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 333 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:17.078338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d7647-b8f4-4538-b923-fcc61a607757 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:17.078338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d7647-b8f4-4538-b923-fcc61a607757 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:17.078338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d7647-b8f4-4538-b923-fcc61a607757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:17.116961 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b1475-0bf3-4182-a2db-3ae13c6f1ff4 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:17.118694 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8c1b1475-0bf3-4182-a2db-3ae13c6f1ff4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:17.158053 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7d4d7647-b8f4-4538-b923-fcc61a607757 None None] MFA Rules not processed for user `296853a88430438e96e0f8fb97b46d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:17.166657 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7d4d7647-b8f4-4538-b923-fcc61a607757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:17.170068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c1b1475-0bf3-4182-a2db-3ae13c6f1ff4 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:17.198122 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cbdfeba-6a47-4dab-8e63-7e17129a0bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:17.201186 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdfeba-6a47-4dab-8e63-7e17129a0bf9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:17.201474 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdfeba-6a47-4dab-8e63-7e17129a0bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:17.209357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cbdfeba-6a47-4dab-8e63-7e17129a0bf9 admin admin] PATH_INFO: `/v3/projects/8aec473535ea4a208e41a21d422ed1ec` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:17.212129 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbdfeba-6a47-4dab-8e63-7e17129a0bf9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8aec473535ea4a208e41a21d422ed1ec)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:17.223181 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cbdfeba-6a47-4dab-8e63-7e17129a0bf9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:17.468501 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4cbdfeba-6a47-4dab-8e63-7e17129a0bf9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8aec473535ea4a208e41a21d422ed1ec', 'request_id': 'req-4cbdfeba-6a47-4dab-8e63-7e17129a0bf9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:17.486953 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 867/1731] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:17 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 419 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:17.505780 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4cbdfeba-6a47-4dab-8e63-7e17129a0bf9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8aec473535ea4a208e41a21d422ed1ec', 'request_id': 'req-4cbdfeba-6a47-4dab-8e63-7e17129a0bf9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:17.529506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4cbdfeba-6a47-4dab-8e63-7e17129a0bf9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8aec473535ea4a208e41a21d422ed1ec', 'request_id': 'req-4cbdfeba-6a47-4dab-8e63-7e17129a0bf9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:17.591690 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4cbdfeba-6a47-4dab-8e63-7e17129a0bf9 admin admin] The token cache is being invalidate because project 8aec473535ea4a208e41a21d422ed1ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:17.594095 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4cbdfeba-6a47-4dab-8e63-7e17129a0bf9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:17.602804 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4cbdfeba-6a47-4dab-8e63-7e17129a0bf9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:17.610666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 865/1732] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:17 2022] DELETE /identity/v3/projects/8aec473535ea4a208e41a21d422ed1ec => generated 0 bytes in 494 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:17.617290 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145e328-dbbb-4309-b8b1-686691b0dbc2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:17.617882 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145e328-dbbb-4309-b8b1-686691b0dbc2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:17.618401 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3145e328-dbbb-4309-b8b1-686691b0dbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:17.663584 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3145e328-dbbb-4309-b8b1-686691b0dbc2 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:17.665404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3145e328-dbbb-4309-b8b1-686691b0dbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:17.674143 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:17.806695 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773a1ad-194c-4ef2-8f13-6c1dc702ed09 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:17.807104 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773a1ad-194c-4ef2-8f13-6c1dc702ed09 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:17.807773 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e773a1ad-194c-4ef2-8f13-6c1dc702ed09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:17.869196 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e773a1ad-194c-4ef2-8f13-6c1dc702ed09 None None] MFA Rules not processed for user `ce1c85ee44ef45d2b631e51112b5ea0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:17.870696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e773a1ad-194c-4ef2-8f13-6c1dc702ed09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:17.937703 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 868/1733] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:17.939393 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa7aa9-f191-479f-b379-70baf562d38d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:17.940703 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa7aa9-f191-479f-b379-70baf562d38d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:17.942139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fefa7aa9-f191-479f-b379-70baf562d38d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:17.988261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-fefa7aa9-f191-479f-b379-70baf562d38d None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:17.989634 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fefa7aa9-f191-479f-b379-70baf562d38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:18.022229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 869/1734] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:18.023370 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa7aa9-f191-479f-b379-70baf562d38d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:18.030687 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fefa7aa9-f191-479f-b379-70baf562d38d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:18.051211 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 866/1735] 10.222.0.55 () {58 vars in 1015 bytes} [Mon Jul 11 14:07:17 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:18.052287 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773a1ad-194c-4ef2-8f13-6c1dc702ed09 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:18.104964 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e773a1ad-194c-4ef2-8f13-6c1dc702ed09 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:18.119884 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38280012-51b3-4c61-8417-fa4755cc15a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:18.121834 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38280012-51b3-4c61-8417-fa4755cc15a2 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:18.122129 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38280012-51b3-4c61-8417-fa4755cc15a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:18.122445 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38280012-51b3-4c61-8417-fa4755cc15a2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:18.123225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38280012-51b3-4c61-8417-fa4755cc15a2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:07:18.124088 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38280012-51b3-4c61-8417-fa4755cc15a2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:18.126167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38280012-51b3-4c61-8417-fa4755cc15a2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:18.138240 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:18.163552 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 867/1736] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:07:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:07:18.164811 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38280012-51b3-4c61-8417-fa4755cc15a2 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:18.167998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-38280012-51b3-4c61-8417-fa4755cc15a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:18.194625 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38280012-51b3-4c61-8417-fa4755cc15a2 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:18.211356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccfc2f7-8c22-41e7-9211-f3585aed46e2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:18.212434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfc2f7-8c22-41e7-9211-f3585aed46e2 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:18.212880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfc2f7-8c22-41e7-9211-f3585aed46e2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:18.213316 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cccfc2f7-8c22-41e7-9211-f3585aed46e2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:18.221335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fefa7aa9-f191-479f-b379-70baf562d38d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:18.226189 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5ca26-ec09-4f74-b397-6f56b9d95e57 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:18.227058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5ca26-ec09-4f74-b397-6f56b9d95e57 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:18.227223 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccfc2f7-8c22-41e7-9211-f3585aed46e2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:18.227682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5ca26-ec09-4f74-b397-6f56b9d95e57 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:18.229241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a5ca26-ec09-4f74-b397-6f56b9d95e57 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:18.230723 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cccfc2f7-8c22-41e7-9211-f3585aed46e2 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:18.231525 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-29a5ca26-ec09-4f74-b397-6f56b9d95e57 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:18.270650 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 868/1737] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:18.273610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccfc2f7-8c22-41e7-9211-f3585aed46e2 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:18.307433 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cccfc2f7-8c22-41e7-9211-f3585aed46e2 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:18.319709 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b27f4-3a30-4fb2-8f41-c52aa388ae6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:18.320963 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b27f4-3a30-4fb2-8f41-c52aa388ae6e admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:18.323067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b27f4-3a30-4fb2-8f41-c52aa388ae6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:18.323476 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-908b27f4-3a30-4fb2-8f41-c52aa388ae6e admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:18.330003 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b27f4-3a30-4fb2-8f41-c52aa388ae6e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:18.338100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-908b27f4-3a30-4fb2-8f41-c52aa388ae6e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:18.383913 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5ca26-ec09-4f74-b397-6f56b9d95e57 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:18.387924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a5ca26-ec09-4f74-b397-6f56b9d95e57 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:18.443476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 870/1739] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:18 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:18.443706 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 869/1738] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:18 2022] POST /identity/v3/projects => generated 373 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:18.444695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5ca26-ec09-4f74-b397-6f56b9d95e57 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:18.459980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a5ca26-ec09-4f74-b397-6f56b9d95e57 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:18.601063 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e21463-c52d-47aa-ab1a-9d2a6816be48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:18.604200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e21463-c52d-47aa-ab1a-9d2a6816be48 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:18.604802 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e21463-c52d-47aa-ab1a-9d2a6816be48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:18.605916 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99e21463-c52d-47aa-ab1a-9d2a6816be48 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:18.607906 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e21463-c52d-47aa-ab1a-9d2a6816be48 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:18.613544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99e21463-c52d-47aa-ab1a-9d2a6816be48 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:18.639095 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:18.672203 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 871/1740] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:18 2022] POST /identity/v3/users => generated 287 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:18.682590 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b27f4-3a30-4fb2-8f41-c52aa388ae6e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:18.705563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-908b27f4-3a30-4fb2-8f41-c52aa388ae6e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:18.744520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4358d-58b4-4a71-8616-5f34c9cd95d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:18.745394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4358d-58b4-4a71-8616-5f34c9cd95d4 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:18.745835 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4358d-58b4-4a71-8616-5f34c9cd95d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:18.746274 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09f4358d-58b4-4a71-8616-5f34c9cd95d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:18.747225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4358d-58b4-4a71-8616-5f34c9cd95d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:18.765187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09f4358d-58b4-4a71-8616-5f34c9cd95d4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:18.800004 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 870/1741] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 120 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:18.805110 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e21463-c52d-47aa-ab1a-9d2a6816be48 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:18.825086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99e21463-c52d-47aa-ab1a-9d2a6816be48 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:18.844341 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97eea38a-9ae1-4b28-b893-14af80cf694f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:18.846721 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eea38a-9ae1-4b28-b893-14af80cf694f admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:18.849020 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eea38a-9ae1-4b28-b893-14af80cf694f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:18.849443 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97eea38a-9ae1-4b28-b893-14af80cf694f admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:18.852380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eea38a-9ae1-4b28-b893-14af80cf694f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:18.857084 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97eea38a-9ae1-4b28-b893-14af80cf694f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:18.893058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 872/1742] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:18 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:18.900765 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4358d-58b4-4a71-8616-5f34c9cd95d4 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:18.919385 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09f4358d-58b4-4a71-8616-5f34c9cd95d4 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:18.934098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c804b3-8513-4a5b-afd0-7d1c9e3fb73a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:18.937131 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c804b3-8513-4a5b-afd0-7d1c9e3fb73a admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:18.938985 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c804b3-8513-4a5b-afd0-7d1c9e3fb73a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:18.940411 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c804b3-8513-4a5b-afd0-7d1c9e3fb73a admin admin] PATH_INFO: `/v3/projects/fdd5f139308e44958c5eb1d5e710d51c/users/2069ff0aa9464496b42589d735220b9c/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:18.985714 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c804b3-8513-4a5b-afd0-7d1c9e3fb73a admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdd5f139308e44958c5eb1d5e710d51c, user_id=2069ff0aa9464496b42589d735220b9c, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:18.989517 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c804b3-8513-4a5b-afd0-7d1c9e3fb73a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:19.030069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 871/1743] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:18 2022] PUT /identity/v3/projects/fdd5f139308e44958c5eb1d5e710d51c/users/2069ff0aa9464496b42589d735220b9c/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:19.041777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed1eac-f295-48a3-9832-26df20abf636 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:19.042316 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed1eac-f295-48a3-9832-26df20abf636 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:19.042860 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ed1eac-f295-48a3-9832-26df20abf636 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:19.142476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-64ed1eac-f295-48a3-9832-26df20abf636 None None] MFA Rules not processed for user `2069ff0aa9464496b42589d735220b9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:19.144225 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-64ed1eac-f295-48a3-9832-26df20abf636 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:19.279336 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 873/1744] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:19 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 239 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:19.284228 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c804b3-8513-4a5b-afd0-7d1c9e3fb73a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:19.472865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c804b3-8513-4a5b-afd0-7d1c9e3fb73a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:19.488242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7d4d6-7e7a-43b2-9bb2-8fa5d994e589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:19.491097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7d4d6-7e7a-43b2-9bb2-8fa5d994e589 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:19.491485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7d4d6-7e7a-43b2-9bb2-8fa5d994e589 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:19.494740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7d4d6-7e7a-43b2-9bb2-8fa5d994e589 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:19.496765 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7d4d6-7e7a-43b2-9bb2-8fa5d994e589 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:19.506996 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7d4d6-7e7a-43b2-9bb2-8fa5d994e589 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:19.604084 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed1eac-f295-48a3-9832-26df20abf636 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:19.619267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 872/1745] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:19 2022] POST /identity/v3/projects => generated 373 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:07:19.629161 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ed1eac-f295-48a3-9832-26df20abf636 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:19.836500 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3414b-468d-43b8-9f97-708da02095d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:19.837881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3414b-468d-43b8-9f97-708da02095d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:19.837881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3414b-468d-43b8-9f97-708da02095d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:19.837881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e3414b-468d-43b8-9f97-708da02095d8 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:19.841311 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3414b-468d-43b8-9f97-708da02095d8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:19.843873 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93e3414b-468d-43b8-9f97-708da02095d8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:19.900700 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 874/1746] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:19 2022] POST /identity/v3/users => generated 293 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:19.902831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7d4d6-7e7a-43b2-9bb2-8fa5d994e589 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:19.919264 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7d4d6-7e7a-43b2-9bb2-8fa5d994e589 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:19.933731 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649e597-9565-4d26-9f56-6fadef7e15f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:19.934541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649e597-9565-4d26-9f56-6fadef7e15f3 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:19.936401 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649e597-9565-4d26-9f56-6fadef7e15f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:19.936714 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5649e597-9565-4d26-9f56-6fadef7e15f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:19.937515 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649e597-9565-4d26-9f56-6fadef7e15f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:19.940394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5649e597-9565-4d26-9f56-6fadef7e15f3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:19.987293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 873/1747] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:19 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:19.991197 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3414b-468d-43b8-9f97-708da02095d8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:20.009626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e3414b-468d-43b8-9f97-708da02095d8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:20.022133 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18002bc-3bb1-4036-9926-4e0bfec176ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:20.022779 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18002bc-3bb1-4036-9926-4e0bfec176ad admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:20.023174 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18002bc-3bb1-4036-9926-4e0bfec176ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:20.023353 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18002bc-3bb1-4036-9926-4e0bfec176ad admin admin] PATH_INFO: `/v3/projects/af01d0ef80f74ab1abd6f8df667ee76a/users/563c8fc5cca344adb97d7a106dff49e8/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:20.060453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18002bc-3bb1-4036-9926-4e0bfec176ad admin admin] RBAC: Authorizing `identity:create_grant(project_id=af01d0ef80f74ab1abd6f8df667ee76a, user_id=563c8fc5cca344adb97d7a106dff49e8, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:20.063454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18002bc-3bb1-4036-9926-4e0bfec176ad admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:20.094783 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 875/1748] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:19 2022] PUT /identity/v3/projects/af01d0ef80f74ab1abd6f8df667ee76a/users/563c8fc5cca344adb97d7a106dff49e8/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:20.099018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649e597-9565-4d26-9f56-6fadef7e15f3 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:20.119395 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5649e597-9565-4d26-9f56-6fadef7e15f3 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:20.379897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f4a81b-99bc-4273-9159-9c8d35f118ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:20.380626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4a81b-99bc-4273-9159-9c8d35f118ea admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:20.385209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4a81b-99bc-4273-9159-9c8d35f118ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:20.385428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f4a81b-99bc-4273-9159-9c8d35f118ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:20.392404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4a81b-99bc-4273-9159-9c8d35f118ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:20.398019 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77f4a81b-99bc-4273-9159-9c8d35f118ea admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:20.402438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:20.421697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18002bc-3bb1-4036-9926-4e0bfec176ad admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:20.422183 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 874/1749] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:20 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:20.451348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18002bc-3bb1-4036-9926-4e0bfec176ad admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:20.460132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2f30f-6a09-42d2-bb59-682d36e7a15a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:20.464664 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2f30f-6a09-42d2-bb59-682d36e7a15a admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:20.465439 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2f30f-6a09-42d2-bb59-682d36e7a15a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:20.466207 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd2f30f-6a09-42d2-bb59-682d36e7a15a admin admin] PATH_INFO: `/v3/projects/af01d0ef80f74ab1abd6f8df667ee76a/users/563c8fc5cca344adb97d7a106dff49e8/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:20.469964 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2f30f-6a09-42d2-bb59-682d36e7a15a admin admin] RBAC: Authorizing `identity:create_grant(project_id=af01d0ef80f74ab1abd6f8df667ee76a, user_id=563c8fc5cca344adb97d7a106dff49e8, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:20.473028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd2f30f-6a09-42d2-bb59-682d36e7a15a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:20.525515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 876/1750] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:20 2022] PUT /identity/v3/projects/af01d0ef80f74ab1abd6f8df667ee76a/users/563c8fc5cca344adb97d7a106dff49e8/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:20.538363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0b460-3f76-465b-9c62-1a0d8a16e72e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:20.538912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0b460-3f76-465b-9c62-1a0d8a16e72e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:20.539267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2f0b460-3f76-465b-9c62-1a0d8a16e72e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:20.657252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a2f0b460-3f76-465b-9c62-1a0d8a16e72e None None] MFA Rules not processed for user `563c8fc5cca344adb97d7a106dff49e8`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:20.664177 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a2f0b460-3f76-465b-9c62-1a0d8a16e72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:20.939698 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 875/1751] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:20 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 404 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:20.950392 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2f30f-6a09-42d2-bb59-682d36e7a15a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:20.994482 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:21.097430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd2f30f-6a09-42d2-bb59-682d36e7a15a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:21.105763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815af6e1-432c-480f-9fd9-4a7bb612a0f5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:21.106493 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815af6e1-432c-480f-9fd9-4a7bb612a0f5 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:21.106778 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815af6e1-432c-480f-9fd9-4a7bb612a0f5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:21.107056 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-815af6e1-432c-480f-9fd9-4a7bb612a0f5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:21.126268 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815af6e1-432c-480f-9fd9-4a7bb612a0f5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:21.127774 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-815af6e1-432c-480f-9fd9-4a7bb612a0f5 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:21.154450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 877/1752] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:20 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:21.223060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0b460-3f76-465b-9c62-1a0d8a16e72e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:21.224836 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a2f0b460-3f76-465b-9c62-1a0d8a16e72e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:21.416941 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2f0b460-3f76-465b-9c62-1a0d8a16e72e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:21.433843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a011a-1089-43ca-99a9-d336a23147ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:21.433843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a011a-1089-43ca-99a9-d336a23147ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:21.433843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a011a-1089-43ca-99a9-d336a23147ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:21.433843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d3a011a-1089-43ca-99a9-d336a23147ce admin admin] PATH_INFO: `/v3/projects/036d8b8abfd044b1b53efde9ba5854b9` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:21.442086 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a011a-1089-43ca-99a9-d336a23147ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=036d8b8abfd044b1b53efde9ba5854b9)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:21.446944 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d3a011a-1089-43ca-99a9-d336a23147ce admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:21.596767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9d3a011a-1089-43ca-99a9-d336a23147ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '036d8b8abfd044b1b53efde9ba5854b9', 'request_id': 'req-9d3a011a-1089-43ca-99a9-d336a23147ce'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:21.620633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9d3a011a-1089-43ca-99a9-d336a23147ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '036d8b8abfd044b1b53efde9ba5854b9', 'request_id': 'req-9d3a011a-1089-43ca-99a9-d336a23147ce'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:21.638787 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9d3a011a-1089-43ca-99a9-d336a23147ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '036d8b8abfd044b1b53efde9ba5854b9', 'request_id': 'req-9d3a011a-1089-43ca-99a9-d336a23147ce'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:21.688558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9d3a011a-1089-43ca-99a9-d336a23147ce admin admin] The token cache is being invalidate because project 036d8b8abfd044b1b53efde9ba5854b9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:21.688919 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9d3a011a-1089-43ca-99a9-d336a23147ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:21.697607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9d3a011a-1089-43ca-99a9-d336a23147ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:21.703150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 876/1753] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:21 2022] DELETE /identity/v3/projects/036d8b8abfd044b1b53efde9ba5854b9 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:21.707529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815af6e1-432c-480f-9fd9-4a7bb612a0f5 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:21.736462 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-815af6e1-432c-480f-9fd9-4a7bb612a0f5 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:21.986939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da46f8d-b527-4add-8421-5eb86d20607b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:21.988197 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da46f8d-b527-4add-8421-5eb86d20607b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:21.988197 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da46f8d-b527-4add-8421-5eb86d20607b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:21.992695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da46f8d-b527-4add-8421-5eb86d20607b admin admin] PATH_INFO: `/v3/users/cc5a9253b1624b2ebfaae540ba73f768` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:21.995869 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da46f8d-b527-4add-8421-5eb86d20607b admin admin] RBAC: Authorizing `identity:delete_user(user_id=cc5a9253b1624b2ebfaae540ba73f768)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:21.999765 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da46f8d-b527-4add-8421-5eb86d20607b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:22.179925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5da46f8d-b527-4add-8421-5eb86d20607b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cc5a9253b1624b2ebfaae540ba73f768'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:22.182201 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5da46f8d-b527-4add-8421-5eb86d20607b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cc5a9253b1624b2ebfaae540ba73f768'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:22.247689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5da46f8d-b527-4add-8421-5eb86d20607b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cc5a9253b1624b2ebfaae540ba73f768'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:22.249440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5da46f8d-b527-4add-8421-5eb86d20607b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cc5a9253b1624b2ebfaae540ba73f768'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:22.266596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a011a-1089-43ca-99a9-d336a23147ce admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:22.268185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9d3a011a-1089-43ca-99a9-d336a23147ce admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:22.308691 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d3a011a-1089-43ca-99a9-d336a23147ce admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:22.325326 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb45e72-90b7-4ef2-afc2-24c4c6476b77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:22.328553 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb45e72-90b7-4ef2-afc2-24c4c6476b77 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:22.339661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb45e72-90b7-4ef2-afc2-24c4c6476b77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:22.339791 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5da46f8d-b527-4add-8421-5eb86d20607b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cc5a9253b1624b2ebfaae540ba73f768'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:22.340212 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bb45e72-90b7-4ef2-afc2-24c4c6476b77 admin admin] PATH_INFO: `/v3/users/2069ff0aa9464496b42589d735220b9c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:22.342682 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb45e72-90b7-4ef2-afc2-24c4c6476b77 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2069ff0aa9464496b42589d735220b9c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:22.352952 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bb45e72-90b7-4ef2-afc2-24c4c6476b77 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:22.367853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:22.424495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 878/1754] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:21 2022] DELETE /identity/v3/users/cc5a9253b1624b2ebfaae540ba73f768 => generated 0 bytes in 689 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:22.514227 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4bb45e72-90b7-4ef2-afc2-24c4c6476b77 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2069ff0aa9464496b42589d735220b9c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:22.558594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4bb45e72-90b7-4ef2-afc2-24c4c6476b77 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2069ff0aa9464496b42589d735220b9c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:22.580804 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4bb45e72-90b7-4ef2-afc2-24c4c6476b77 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2069ff0aa9464496b42589d735220b9c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:22.581337 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4bb45e72-90b7-4ef2-afc2-24c4c6476b77 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2069ff0aa9464496b42589d735220b9c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:22.607133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4bb45e72-90b7-4ef2-afc2-24c4c6476b77 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2069ff0aa9464496b42589d735220b9c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:22.628380 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 877/1755] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:22 2022] DELETE /identity/v3/users/2069ff0aa9464496b42589d735220b9c => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:22.631029 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7079a6-38e2-44f2-ae9f-02e43f07ff64 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:22.631342 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7079a6-38e2-44f2-ae9f-02e43f07ff64 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:22.631679 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b7079a6-38e2-44f2-ae9f-02e43f07ff64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:22.709111 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3b7079a6-38e2-44f2-ae9f-02e43f07ff64 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:22.716660 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3b7079a6-38e2-44f2-ae9f-02e43f07ff64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:22.832276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:22.888707 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb45e72-90b7-4ef2-afc2-24c4c6476b77 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:22.890176 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4bb45e72-90b7-4ef2-afc2-24c4c6476b77 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:22.901092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 879/1756] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 272 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:22.926855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bb45e72-90b7-4ef2-afc2-24c4c6476b77 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:22.960056 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977f7a58-d4fd-449f-b699-5a4e22471259 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:22.960971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f7a58-d4fd-449f-b699-5a4e22471259 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:22.961372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f7a58-d4fd-449f-b699-5a4e22471259 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:22.961550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977f7a58-d4fd-449f-b699-5a4e22471259 admin admin] PATH_INFO: `/v3/users/09ab39f9e6b64b35ab050d3e4380b80f` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:22.978423 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f7a58-d4fd-449f-b699-5a4e22471259 admin admin] RBAC: Authorizing `identity:delete_user(user_id=09ab39f9e6b64b35ab050d3e4380b80f)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:22.984151 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977f7a58-d4fd-449f-b699-5a4e22471259 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:23.050347 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7079a6-38e2-44f2-ae9f-02e43f07ff64 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:23.050347 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3b7079a6-38e2-44f2-ae9f-02e43f07ff64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:23.174831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-977f7a58-d4fd-449f-b699-5a4e22471259 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '09ab39f9e6b64b35ab050d3e4380b80f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:23.236488 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-977f7a58-d4fd-449f-b699-5a4e22471259 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '09ab39f9e6b64b35ab050d3e4380b80f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:23.245586 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b7079a6-38e2-44f2-ae9f-02e43f07ff64 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:23.261017 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f41a1b-0a27-499a-9d20-f4111fe456d4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:23.264992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f41a1b-0a27-499a-9d20-f4111fe456d4 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:23.265431 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f41a1b-0a27-499a-9d20-f4111fe456d4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:23.265580 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63f41a1b-0a27-499a-9d20-f4111fe456d4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:23.268505 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-977f7a58-d4fd-449f-b699-5a4e22471259 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '09ab39f9e6b64b35ab050d3e4380b80f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:23.269645 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-977f7a58-d4fd-449f-b699-5a4e22471259 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '09ab39f9e6b64b35ab050d3e4380b80f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:23.311687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-977f7a58-d4fd-449f-b699-5a4e22471259 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '09ab39f9e6b64b35ab050d3e4380b80f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:23.335619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 878/1757] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:22 2022] DELETE /identity/v3/users/09ab39f9e6b64b35ab050d3e4380b80f => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:23.339792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8dd90-1278-4d9b-bdf5-4961fabe2097 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:23.341672 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8dd90-1278-4d9b-bdf5-4961fabe2097 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:23.344150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfb8dd90-1278-4d9b-bdf5-4961fabe2097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:23.354071 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f41a1b-0a27-499a-9d20-f4111fe456d4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:23.376561 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63f41a1b-0a27-499a-9d20-f4111fe456d4 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:23.419261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-cfb8dd90-1278-4d9b-bdf5-4961fabe2097 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:23.423524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-cfb8dd90-1278-4d9b-bdf5-4961fabe2097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:23.448470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 880/1758] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:07:23.627268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:23.661510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 879/1759] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 323 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:23.973482 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f41a1b-0a27-499a-9d20-f4111fe456d4 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:23.986499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63f41a1b-0a27-499a-9d20-f4111fe456d4 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:24.235153 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce869ab7-c785-4566-8544-d05d1dda0ccd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:24.238053 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce869ab7-c785-4566-8544-d05d1dda0ccd service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:24.239729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce869ab7-c785-4566-8544-d05d1dda0ccd service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:24.242725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce869ab7-c785-4566-8544-d05d1dda0ccd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:24.283874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce869ab7-c785-4566-8544-d05d1dda0ccd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:24.286636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce869ab7-c785-4566-8544-d05d1dda0ccd service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:24.320043 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 881/1760] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:25.618697 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08446ad-617a-461b-b881-aca9c83b7d65 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:25.619337 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08446ad-617a-461b-b881-aca9c83b7d65 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:25.619511 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08446ad-617a-461b-b881-aca9c83b7d65 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:25.621246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 880/1761] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 11 14:07:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:07:25.679597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bd6ab-9ae6-462b-ba77-20845b9f4cbb None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:25.679597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bd6ab-9ae6-462b-ba77-20845b9f4cbb None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:25.679597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3bd6ab-9ae6-462b-ba77-20845b9f4cbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:25.738221 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ee3bd6ab-9ae6-462b-ba77-20845b9f4cbb None None] MFA Rules not processed for user `ef5dac3f311e43809a46995bab56e780`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:25.740043 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ee3bd6ab-9ae6-462b-ba77-20845b9f4cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:25.965512 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 882/1762] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 11 14:07:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 289 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:26.316102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08446ad-617a-461b-b881-aca9c83b7d65 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:26.318021 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d08446ad-617a-461b-b881-aca9c83b7d65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:26.351935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08446ad-617a-461b-b881-aca9c83b7d65 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:26.357595 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1060351-d121-43fd-a418-731ab0405283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:26.358522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1060351-d121-43fd-a418-731ab0405283 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:26.358813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1060351-d121-43fd-a418-731ab0405283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:26.359097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1060351-d121-43fd-a418-731ab0405283 admin admin] PATH_INFO: `/v3/projects/fdd5f139308e44958c5eb1d5e710d51c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:26.360393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1060351-d121-43fd-a418-731ab0405283 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdd5f139308e44958c5eb1d5e710d51c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:26.362397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1060351-d121-43fd-a418-731ab0405283 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:26.373353 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bd6ab-9ae6-462b-ba77-20845b9f4cbb None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:26.379512 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ee3bd6ab-9ae6-462b-ba77-20845b9f4cbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:26.415266 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3bd6ab-9ae6-462b-ba77-20845b9f4cbb None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:26.426560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f49e093-f1de-4ae6-8491-a769c669ea51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:26.427720 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49e093-f1de-4ae6-8491-a769c669ea51 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:26.428014 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49e093-f1de-4ae6-8491-a769c669ea51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:26.428297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f49e093-f1de-4ae6-8491-a769c669ea51 admin admin] PATH_INFO: `/v3/projects/6f60d4a9b9b0474883a1b58607111c6d` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:26.433552 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49e093-f1de-4ae6-8491-a769c669ea51 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f60d4a9b9b0474883a1b58607111c6d)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:26.437697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f49e093-f1de-4ae6-8491-a769c669ea51 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:26.513154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:26.524054 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a1060351-d121-43fd-a418-731ab0405283 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdd5f139308e44958c5eb1d5e710d51c', 'request_id': 'req-a1060351-d121-43fd-a418-731ab0405283'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:26.548604 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a1060351-d121-43fd-a418-731ab0405283 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdd5f139308e44958c5eb1d5e710d51c', 'request_id': 'req-a1060351-d121-43fd-a418-731ab0405283'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:26.575260 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a1060351-d121-43fd-a418-731ab0405283 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdd5f139308e44958c5eb1d5e710d51c', 'request_id': 'req-a1060351-d121-43fd-a418-731ab0405283'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:26.618948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6f49e093-f1de-4ae6-8491-a769c669ea51 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f60d4a9b9b0474883a1b58607111c6d', 'request_id': 'req-6f49e093-f1de-4ae6-8491-a769c669ea51'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:26.638044 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a1060351-d121-43fd-a418-731ab0405283 admin admin] The token cache is being invalidate because project fdd5f139308e44958c5eb1d5e710d51c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:26.638520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a1060351-d121-43fd-a418-731ab0405283 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:26.641483 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6f49e093-f1de-4ae6-8491-a769c669ea51 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f60d4a9b9b0474883a1b58607111c6d', 'request_id': 'req-6f49e093-f1de-4ae6-8491-a769c669ea51'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:26.643433 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a1060351-d121-43fd-a418-731ab0405283 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:26.649851 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 881/1763] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:26 2022] DELETE /identity/v3/projects/fdd5f139308e44958c5eb1d5e710d51c => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:26.658790 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1060351-d121-43fd-a418-731ab0405283 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:26.660453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a1060351-d121-43fd-a418-731ab0405283 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:26.678681 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6f49e093-f1de-4ae6-8491-a769c669ea51 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f60d4a9b9b0474883a1b58607111c6d', 'request_id': 'req-6f49e093-f1de-4ae6-8491-a769c669ea51'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:26.752853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6f49e093-f1de-4ae6-8491-a769c669ea51 admin admin] The token cache is being invalidate because project 6f60d4a9b9b0474883a1b58607111c6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:26.753685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6f49e093-f1de-4ae6-8491-a769c669ea51 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:26.758835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6f49e093-f1de-4ae6-8491-a769c669ea51 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:26.762873 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 883/1764] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:26 2022] DELETE /identity/v3/projects/6f60d4a9b9b0474883a1b58607111c6d => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:26.770353 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e778eb-1929-437b-84ac-4ece9fb7b03f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:26.770802 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e778eb-1929-437b-84ac-4ece9fb7b03f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:26.771095 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e778eb-1929-437b-84ac-4ece9fb7b03f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:26.850646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-93e778eb-1929-437b-84ac-4ece9fb7b03f None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:26.856410 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-93e778eb-1929-437b-84ac-4ece9fb7b03f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:27.005886 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1060351-d121-43fd-a418-731ab0405283 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:27.110025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 884/1765] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:27.110025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f5c3c-7297-430f-88d9-b72a6028d7a7 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:27.111136 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f5c3c-7297-430f-88d9-b72a6028d7a7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:27.111466 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f72f5c3c-7297-430f-88d9-b72a6028d7a7 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:27.127301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 885/1766] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 11 14:07:27 2022] GET /identity => generated 270 bytes in 20 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:07:27.128460 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc259f75-ee08-4067-81bc-0b7947938f00 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:27.128759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc259f75-ee08-4067-81bc-0b7947938f00 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:27.129044 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc259f75-ee08-4067-81bc-0b7947938f00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:27.209456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9aeaf-62f8-4222-98c6-4eabde68a919 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:27.210636 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9aeaf-62f8-4222-98c6-4eabde68a919 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:27.210930 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9aeaf-62f8-4222-98c6-4eabde68a919 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:27.211434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bde9aeaf-62f8-4222-98c6-4eabde68a919 admin admin] PATH_INFO: `/v3/users/563c8fc5cca344adb97d7a106dff49e8` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:27.213035 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9aeaf-62f8-4222-98c6-4eabde68a919 admin admin] RBAC: Authorizing `identity:delete_user(user_id=563c8fc5cca344adb97d7a106dff49e8)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:27.217059 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bde9aeaf-62f8-4222-98c6-4eabde68a919 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:27.219932 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-bc259f75-ee08-4067-81bc-0b7947938f00 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:27.221293 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bc259f75-ee08-4067-81bc-0b7947938f00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:27.263287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 886/1767] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:27.263287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a646fe4-aae2-44e4-b865-59dfaa7d7aa6 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:27.263287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a646fe4-aae2-44e4-b865-59dfaa7d7aa6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:27.266965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a646fe4-aae2-44e4-b865-59dfaa7d7aa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:27.320225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:27.329375 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7a646fe4-aae2-44e4-b865-59dfaa7d7aa6 None None] MFA Rules not processed for user `ef5dac3f311e43809a46995bab56e780`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:27.348594 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7a646fe4-aae2-44e4-b865-59dfaa7d7aa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:27.354359 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bde9aeaf-62f8-4222-98c6-4eabde68a919 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '563c8fc5cca344adb97d7a106dff49e8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:27.384248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bde9aeaf-62f8-4222-98c6-4eabde68a919 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '563c8fc5cca344adb97d7a106dff49e8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:27.402276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bde9aeaf-62f8-4222-98c6-4eabde68a919 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '563c8fc5cca344adb97d7a106dff49e8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:27.403937 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bde9aeaf-62f8-4222-98c6-4eabde68a919 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '563c8fc5cca344adb97d7a106dff49e8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:27.432688 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bde9aeaf-62f8-4222-98c6-4eabde68a919 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '563c8fc5cca344adb97d7a106dff49e8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:27.448510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 882/1768] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:26 2022] DELETE /identity/v3/users/563c8fc5cca344adb97d7a106dff49e8 => generated 0 bytes in 789 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:27.453949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9aeaf-62f8-4222-98c6-4eabde68a919 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:27.459511 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bde9aeaf-62f8-4222-98c6-4eabde68a919 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:27.593240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 887/1769] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 11 14:07:27 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 332 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:27.662160 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bde9aeaf-62f8-4222-98c6-4eabde68a919 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:27.670037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40822d9e-407a-4e37-bd87-cdb157f9a06f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:27.673780 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40822d9e-407a-4e37-bd87-cdb157f9a06f admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:27.674261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40822d9e-407a-4e37-bd87-cdb157f9a06f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:27.674810 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40822d9e-407a-4e37-bd87-cdb157f9a06f admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:27.675860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40822d9e-407a-4e37-bd87-cdb157f9a06f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:07:27.676387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40822d9e-407a-4e37-bd87-cdb157f9a06f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:27.678952 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40822d9e-407a-4e37-bd87-cdb157f9a06f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:27.701494 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 883/1770] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:07:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:07:27.707592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a646fe4-aae2-44e4-b865-59dfaa7d7aa6 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:27.722125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a646fe4-aae2-44e4-b865-59dfaa7d7aa6 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:27.727198 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071746e-1722-4a7c-a915-3985acfb9072 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:27.728236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071746e-1722-4a7c-a915-3985acfb9072 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:27.728236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071746e-1722-4a7c-a915-3985acfb9072 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:27.728434 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b071746e-1722-4a7c-a915-3985acfb9072 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:27.729138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071746e-1722-4a7c-a915-3985acfb9072 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:27.733019 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b071746e-1722-4a7c-a915-3985acfb9072 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:27.811997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 888/1771] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:27 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:27.813298 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40822d9e-407a-4e37-bd87-cdb157f9a06f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:27.833492 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40822d9e-407a-4e37-bd87-cdb157f9a06f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:27.846327 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071746e-1722-4a7c-a915-3985acfb9072 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:27.848195 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b071746e-1722-4a7c-a915-3985acfb9072 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:27.863501 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:28.034414 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951623ed-1251-4ca7-a965-9c1886457f10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:28.036933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951623ed-1251-4ca7-a965-9c1886457f10 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:28.037718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951623ed-1251-4ca7-a965-9c1886457f10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:28.038875 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-951623ed-1251-4ca7-a965-9c1886457f10 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:28.040470 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951623ed-1251-4ca7-a965-9c1886457f10 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:28.043979 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-951623ed-1251-4ca7-a965-9c1886457f10 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:28.073925 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 884/1772] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:27 2022] POST /identity/v3/users => generated 292 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:28.075181 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951623ed-1251-4ca7-a965-9c1886457f10 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:28.077560 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-951623ed-1251-4ca7-a965-9c1886457f10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:28.094544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-951623ed-1251-4ca7-a965-9c1886457f10 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:28.101701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2c4e7-15bf-4b1f-9754-816ec1cb4717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:28.102954 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2c4e7-15bf-4b1f-9754-816ec1cb4717 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:28.103757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2c4e7-15bf-4b1f-9754-816ec1cb4717 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:28.105805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e2c4e7-15bf-4b1f-9754-816ec1cb4717 admin admin] PATH_INFO: `/v3/projects/ac60d6c44bda4b7cb2fbfe80ebb873d1` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:28.107597 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2c4e7-15bf-4b1f-9754-816ec1cb4717 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac60d6c44bda4b7cb2fbfe80ebb873d1)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:28.110477 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e2c4e7-15bf-4b1f-9754-816ec1cb4717 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:28.113533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b071746e-1722-4a7c-a915-3985acfb9072 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:28.133603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c826ae-b044-4dbe-a99d-aea4ca2077ad service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:28.135310 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c826ae-b044-4dbe-a99d-aea4ca2077ad service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:28.137762 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c826ae-b044-4dbe-a99d-aea4ca2077ad service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:28.138740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5c826ae-b044-4dbe-a99d-aea4ca2077ad service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:28.143683 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c5c826ae-b044-4dbe-a99d-aea4ca2077ad service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:28.174073 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=90,keystone:DELETE=6,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:28.176316 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c826ae-b044-4dbe-a99d-aea4ca2077ad service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:28.180167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5c826ae-b044-4dbe-a99d-aea4ca2077ad service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:28.208116 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c0e2c4e7-15bf-4b1f-9754-816ec1cb4717 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac60d6c44bda4b7cb2fbfe80ebb873d1', 'request_id': 'req-c0e2c4e7-15bf-4b1f-9754-816ec1cb4717'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:28.223538 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 889/1773] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 377 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:07:28.224817 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c826ae-b044-4dbe-a99d-aea4ca2077ad service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:28.232882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c0e2c4e7-15bf-4b1f-9754-816ec1cb4717 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac60d6c44bda4b7cb2fbfe80ebb873d1', 'request_id': 'req-c0e2c4e7-15bf-4b1f-9754-816ec1cb4717'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:28.253526 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5c826ae-b044-4dbe-a99d-aea4ca2077ad service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:28.256322 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c0e2c4e7-15bf-4b1f-9754-816ec1cb4717 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac60d6c44bda4b7cb2fbfe80ebb873d1', 'request_id': 'req-c0e2c4e7-15bf-4b1f-9754-816ec1cb4717'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:28.301171 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c0e2c4e7-15bf-4b1f-9754-816ec1cb4717 admin admin] The token cache is being invalidate because project ac60d6c44bda4b7cb2fbfe80ebb873d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:28.301495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c0e2c4e7-15bf-4b1f-9754-816ec1cb4717 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:28.304054 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c0e2c4e7-15bf-4b1f-9754-816ec1cb4717 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:28.313603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 885/1774] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:28 2022] DELETE /identity/v3/projects/ac60d6c44bda4b7cb2fbfe80ebb873d1 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:28.317703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2c4e7-15bf-4b1f-9754-816ec1cb4717 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:28.319190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c0e2c4e7-15bf-4b1f-9754-816ec1cb4717 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:28.406202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b06481-caaa-4444-83b7-dcb3be12b2a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:28.407522 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b06481-caaa-4444-83b7-dcb3be12b2a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:28.407816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b06481-caaa-4444-83b7-dcb3be12b2a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:28.408238 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b06481-caaa-4444-83b7-dcb3be12b2a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:28.408992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b06481-caaa-4444-83b7-dcb3be12b2a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:28.410675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b06481-caaa-4444-83b7-dcb3be12b2a5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:28.430191 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e2c4e7-15bf-4b1f-9754-816ec1cb4717 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:28.441755 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 890/1775] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:28.446987 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da086cf4-eb37-497e-9e6a-213fe91142b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:28.447908 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b06481-caaa-4444-83b7-dcb3be12b2a5 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:28.448165 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da086cf4-eb37-497e-9e6a-213fe91142b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:28.448464 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da086cf4-eb37-497e-9e6a-213fe91142b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:28.448748 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da086cf4-eb37-497e-9e6a-213fe91142b3 admin admin] PATH_INFO: `/v3/users/f8f871a35d44447289969e4afee7f9fb` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:28.451427 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-22b06481-caaa-4444-83b7-dcb3be12b2a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:28.458945 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da086cf4-eb37-497e-9e6a-213fe91142b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f8f871a35d44447289969e4afee7f9fb)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:28.461346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da086cf4-eb37-497e-9e6a-213fe91142b3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:28.495011 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b06481-caaa-4444-83b7-dcb3be12b2a5 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:28.508418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88c1c6-b11d-405a-836b-d9be785f82bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:28.509245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88c1c6-b11d-405a-836b-d9be785f82bb admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:28.509580 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88c1c6-b11d-405a-836b-d9be785f82bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:28.509872 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c88c1c6-b11d-405a-836b-d9be785f82bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:28.510847 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88c1c6-b11d-405a-836b-d9be785f82bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:28.513904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c88c1c6-b11d-405a-836b-d9be785f82bb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:28.536484 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 891/1776] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:28 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:28.538091 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88c1c6-b11d-405a-836b-d9be785f82bb admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:28.563768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-da086cf4-eb37-497e-9e6a-213fe91142b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f8f871a35d44447289969e4afee7f9fb'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:28.579737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c88c1c6-b11d-405a-836b-d9be785f82bb admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:28.591253 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-da086cf4-eb37-497e-9e6a-213fe91142b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f8f871a35d44447289969e4afee7f9fb'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:28.597147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d54eb-f59c-491e-8163-bac618f1abe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:28.598532 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d54eb-f59c-491e-8163-bac618f1abe9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:28.598953 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d54eb-f59c-491e-8163-bac618f1abe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:28.601102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d54eb-f59c-491e-8163-bac618f1abe9 admin admin] PATH_INFO: `/v3/projects/cad3ea44ef7f421291cedcde0bf2d468/users/17285d4bb61245b19331aa0c915a5e4b/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:28.601548 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-da086cf4-eb37-497e-9e6a-213fe91142b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f8f871a35d44447289969e4afee7f9fb'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:28.602284 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-da086cf4-eb37-497e-9e6a-213fe91142b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f8f871a35d44447289969e4afee7f9fb'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:28.621779 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-da086cf4-eb37-497e-9e6a-213fe91142b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f8f871a35d44447289969e4afee7f9fb'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:28.648145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d54eb-f59c-491e-8163-bac618f1abe9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cad3ea44ef7f421291cedcde0bf2d468, user_id=17285d4bb61245b19331aa0c915a5e4b, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:28.648576 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d54eb-f59c-491e-8163-bac618f1abe9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:28.648898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 886/1777] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:28 2022] DELETE /identity/v3/users/f8f871a35d44447289969e4afee7f9fb => generated 0 bytes in 329 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:28.689632 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 892/1778] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:28 2022] PUT /identity/v3/projects/cad3ea44ef7f421291cedcde0bf2d468/users/17285d4bb61245b19331aa0c915a5e4b/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:28.694171 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930f071-46cd-4a31-8d5b-1d6f257a9b7b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:28.694886 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930f071-46cd-4a31-8d5b-1d6f257a9b7b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:28.695186 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5930f071-46cd-4a31-8d5b-1d6f257a9b7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:28.759011 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5930f071-46cd-4a31-8d5b-1d6f257a9b7b None None] MFA Rules not processed for user `17285d4bb61245b19331aa0c915a5e4b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:28.760909 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5930f071-46cd-4a31-8d5b-1d6f257a9b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:28.803010 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:28.857232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d54eb-f59c-491e-8163-bac618f1abe9 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:28.859254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8d2d54eb-f59c-491e-8163-bac618f1abe9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:28.901150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 887/1779] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:28 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:28.902854 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930f071-46cd-4a31-8d5b-1d6f257a9b7b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:28.904775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5930f071-46cd-4a31-8d5b-1d6f257a9b7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:28.974282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:29.004405 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d54eb-f59c-491e-8163-bac618f1abe9 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:29.013945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc7302-b6fd-4359-8fa8-22c9a2dd6de9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:29.014218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc7302-b6fd-4359-8fa8-22c9a2dd6de9 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:29.014700 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc7302-b6fd-4359-8fa8-22c9a2dd6de9 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:29.015100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fcc7302-b6fd-4359-8fa8-22c9a2dd6de9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:29.017892 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4fcc7302-b6fd-4359-8fa8-22c9a2dd6de9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:29.035806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc7302-b6fd-4359-8fa8-22c9a2dd6de9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:29.038537 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fcc7302-b6fd-4359-8fa8-22c9a2dd6de9 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:29.064130 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 893/1780] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:28 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:07:29.098963 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5930f071-46cd-4a31-8d5b-1d6f257a9b7b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:29.111839 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045f4bd-c0f8-466b-8b30-d6a9a1f3b89a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:29.112539 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045f4bd-c0f8-466b-8b30-d6a9a1f3b89a admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:29.112825 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045f4bd-c0f8-466b-8b30-d6a9a1f3b89a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:29.113105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a045f4bd-c0f8-466b-8b30-d6a9a1f3b89a admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:29.113817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045f4bd-c0f8-466b-8b30-d6a9a1f3b89a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:29.119445 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a045f4bd-c0f8-466b-8b30-d6a9a1f3b89a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:29.184208 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 888/1781] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:28 2022] POST /identity/v3/projects => generated 383 bytes in 282 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:29.189419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc7302-b6fd-4359-8fa8-22c9a2dd6de9 service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:29.212380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fcc7302-b6fd-4359-8fa8-22c9a2dd6de9 service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:29.303178 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045f4bd-c0f8-466b-8b30-d6a9a1f3b89a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:29.305131 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a045f4bd-c0f8-466b-8b30-d6a9a1f3b89a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:29.354156 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cff97a-c83b-48e5-8d52-0878ccdba067 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:29.355237 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cff97a-c83b-48e5-8d52-0878ccdba067 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:29.356120 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cff97a-c83b-48e5-8d52-0878ccdba067 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:29.356569 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cff97a-c83b-48e5-8d52-0878ccdba067 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:29.357639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cff97a-c83b-48e5-8d52-0878ccdba067 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:29.360445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cff97a-c83b-48e5-8d52-0878ccdba067 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:29.420896 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a045f4bd-c0f8-466b-8b30-d6a9a1f3b89a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:29.426649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898153c-3cd8-4b88-a968-d3f7fd2be021 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:29.427989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898153c-3cd8-4b88-a968-d3f7fd2be021 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:29.428439 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898153c-3cd8-4b88-a968-d3f7fd2be021 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:29.429019 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9898153c-3cd8-4b88-a968-d3f7fd2be021 admin admin] PATH_INFO: `/v3/projects/6355bd126c224f3b9266dcd421fe0da3` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:29.435374 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 894/1782] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:29 2022] POST /identity/v3/users => generated 298 bytes in 244 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:07:29.435374 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cff97a-c83b-48e5-8d52-0878ccdba067 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:29.437057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898153c-3cd8-4b88-a968-d3f7fd2be021 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6355bd126c224f3b9266dcd421fe0da3)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:29.438140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a7cff97a-c83b-48e5-8d52-0878ccdba067 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:29.439953 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9898153c-3cd8-4b88-a968-d3f7fd2be021 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:29.458347 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cff97a-c83b-48e5-8d52-0878ccdba067 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:29.465022 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6cf447-df0f-425a-9d4a-970b97bdb50b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:29.466968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cf447-df0f-425a-9d4a-970b97bdb50b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:29.467559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cf447-df0f-425a-9d4a-970b97bdb50b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:29.467949 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6cf447-df0f-425a-9d4a-970b97bdb50b admin admin] PATH_INFO: `/v3/projects/af01d0ef80f74ab1abd6f8df667ee76a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:29.470240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6cf447-df0f-425a-9d4a-970b97bdb50b admin admin] RBAC: Authorizing `identity:delete_project(project_id=af01d0ef80f74ab1abd6f8df667ee76a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:29.473267 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6cf447-df0f-425a-9d4a-970b97bdb50b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:29.522741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9898153c-3cd8-4b88-a968-d3f7fd2be021 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6355bd126c224f3b9266dcd421fe0da3', 'request_id': 'req-9898153c-3cd8-4b88-a968-d3f7fd2be021'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:29.524557 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:INSERT=1,keystone:DELETE=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:29.536502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9898153c-3cd8-4b88-a968-d3f7fd2be021 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6355bd126c224f3b9266dcd421fe0da3', 'request_id': 'req-9898153c-3cd8-4b88-a968-d3f7fd2be021'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:29.547769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9898153c-3cd8-4b88-a968-d3f7fd2be021 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6355bd126c224f3b9266dcd421fe0da3', 'request_id': 'req-9898153c-3cd8-4b88-a968-d3f7fd2be021'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:29.566715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9898153c-3cd8-4b88-a968-d3f7fd2be021 admin admin] The token cache is being invalidate because project 6355bd126c224f3b9266dcd421fe0da3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:29.567009 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9898153c-3cd8-4b88-a968-d3f7fd2be021 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:29.567743 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9898153c-3cd8-4b88-a968-d3f7fd2be021 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:29.573595 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 889/1783] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:29 2022] DELETE /identity/v3/projects/6355bd126c224f3b9266dcd421fe0da3 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:29.573595 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898153c-3cd8-4b88-a968-d3f7fd2be021 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:29.575393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9898153c-3cd8-4b88-a968-d3f7fd2be021 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:29.636782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ee6cf447-df0f-425a-9d4a-970b97bdb50b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af01d0ef80f74ab1abd6f8df667ee76a', 'request_id': 'req-ee6cf447-df0f-425a-9d4a-970b97bdb50b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:29.655946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ee6cf447-df0f-425a-9d4a-970b97bdb50b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af01d0ef80f74ab1abd6f8df667ee76a', 'request_id': 'req-ee6cf447-df0f-425a-9d4a-970b97bdb50b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:29.673239 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ee6cf447-df0f-425a-9d4a-970b97bdb50b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af01d0ef80f74ab1abd6f8df667ee76a', 'request_id': 'req-ee6cf447-df0f-425a-9d4a-970b97bdb50b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:29.676377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9898153c-3cd8-4b88-a968-d3f7fd2be021 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:29.717228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ee6cf447-df0f-425a-9d4a-970b97bdb50b admin admin] The token cache is being invalidate because project af01d0ef80f74ab1abd6f8df667ee76a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:29.718170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ee6cf447-df0f-425a-9d4a-970b97bdb50b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:29.723099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ee6cf447-df0f-425a-9d4a-970b97bdb50b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:29.730229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 895/1784] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:29 2022] DELETE /identity/v3/projects/af01d0ef80f74ab1abd6f8df667ee76a => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:29.730229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6cf447-df0f-425a-9d4a-970b97bdb50b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:29.743294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ee6cf447-df0f-425a-9d4a-970b97bdb50b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:29.855479 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77957d5d-5996-45a3-9574-f914f64b9bed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:29.856187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77957d5d-5996-45a3-9574-f914f64b9bed admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:29.857792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77957d5d-5996-45a3-9574-f914f64b9bed admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:29.857792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77957d5d-5996-45a3-9574-f914f64b9bed admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:29.867215 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77957d5d-5996-45a3-9574-f914f64b9bed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:29.870685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77957d5d-5996-45a3-9574-f914f64b9bed admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:29.900040 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 890/1785] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:29 2022] GET /identity/v3/roles => generated 1420 bytes in 330 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:29.901832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778947a-4ea9-4894-8609-a06bbe58a333 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:29.903740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778947a-4ea9-4894-8609-a06bbe58a333 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:29.905143 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1778947a-4ea9-4894-8609-a06bbe58a333 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:29.947439 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1778947a-4ea9-4894-8609-a06bbe58a333 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:29.948841 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1778947a-4ea9-4894-8609-a06bbe58a333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:29.962889 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 891/1786] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:29.964068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778947a-4ea9-4894-8609-a06bbe58a333 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:29.966453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1778947a-4ea9-4894-8609-a06bbe58a333 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:29.969047 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:29.990165 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1778947a-4ea9-4894-8609-a06bbe58a333 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:30.007846 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-988c504a-a032-400c-810a-b4f17feacee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:30.008962 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c504a-a032-400c-810a-b4f17feacee2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:30.009395 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c504a-a032-400c-810a-b4f17feacee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:30.009831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-988c504a-a032-400c-810a-b4f17feacee2 admin admin] PATH_INFO: `/v3/projects/0e826af69dec41a19d213faa3b49ec22/users/1ce1a4983f9e4bdbb6c40cf36cd1de0f/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:30.011549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6cf447-df0f-425a-9d4a-970b97bdb50b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:30.032786 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc0f67-53e3-4d2f-9664-07aa899fc8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:30.034489 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc0f67-53e3-4d2f-9664-07aa899fc8d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:30.040533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc0f67-53e3-4d2f-9664-07aa899fc8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:30.041363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc0f67-53e3-4d2f-9664-07aa899fc8d1 admin admin] PATH_INFO: `/v3/users/b13659d602d44f0a986ed8c6acf94ef1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:30.049326 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc0f67-53e3-4d2f-9664-07aa899fc8d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b13659d602d44f0a986ed8c6acf94ef1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:30.050258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c504a-a032-400c-810a-b4f17feacee2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e826af69dec41a19d213faa3b49ec22, user_id=1ce1a4983f9e4bdbb6c40cf36cd1de0f, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:30.056034 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-988c504a-a032-400c-810a-b4f17feacee2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:30.060694 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc0f67-53e3-4d2f-9664-07aa899fc8d1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:30.088309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 892/1787] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:29 2022] PUT /identity/v3/projects/0e826af69dec41a19d213faa3b49ec22/users/1ce1a4983f9e4bdbb6c40cf36cd1de0f/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:30.091959 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c47bfa-6156-4016-95f5-ca5ccd760a26 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:30.092276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c47bfa-6156-4016-95f5-ca5ccd760a26 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:30.092566 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c47bfa-6156-4016-95f5-ca5ccd760a26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:30.144131 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e1c47bfa-6156-4016-95f5-ca5ccd760a26 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:30.145575 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e1c47bfa-6156-4016-95f5-ca5ccd760a26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:30.235176 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-22cc0f67-53e3-4d2f-9664-07aa899fc8d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b13659d602d44f0a986ed8c6acf94ef1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:30.236246 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-22cc0f67-53e3-4d2f-9664-07aa899fc8d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b13659d602d44f0a986ed8c6acf94ef1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:30.269789 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-22cc0f67-53e3-4d2f-9664-07aa899fc8d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b13659d602d44f0a986ed8c6acf94ef1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:30.270597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-22cc0f67-53e3-4d2f-9664-07aa899fc8d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b13659d602d44f0a986ed8c6acf94ef1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:30.310214 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:INSERT=2,keystone:DELETE=3,keystone:SELECT=95 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:30.316985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-22cc0f67-53e3-4d2f-9664-07aa899fc8d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b13659d602d44f0a986ed8c6acf94ef1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:30.364820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 896/1788] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:29 2022] DELETE /identity/v3/users/b13659d602d44f0a986ed8c6acf94ef1 => generated 0 bytes in 637 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:30.372840 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 893/1789] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:30.378531 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc0f67-53e3-4d2f-9664-07aa899fc8d1 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:30.378986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c47bfa-6156-4016-95f5-ca5ccd760a26 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:30.380010 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-22cc0f67-53e3-4d2f-9664-07aa899fc8d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:30.389997 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c47bfa-6156-4016-95f5-ca5ccd760a26 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:30.503950 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc0f67-53e3-4d2f-9664-07aa899fc8d1 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:30.518242 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e6e7d-9f20-41f8-bce8-d1da7b12f2af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:30.519273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38972122-8616-4a5e-ae5c-de665b8587a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:30.520173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e6e7d-9f20-41f8-bce8-d1da7b12f2af admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:30.520876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38972122-8616-4a5e-ae5c-de665b8587a1 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:30.521307 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e6e7d-9f20-41f8-bce8-d1da7b12f2af admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:30.521715 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082e6e7d-9f20-41f8-bce8-d1da7b12f2af admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:30.522074 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38972122-8616-4a5e-ae5c-de665b8587a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:30.523047 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e6e7d-9f20-41f8-bce8-d1da7b12f2af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:30.523509 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38972122-8616-4a5e-ae5c-de665b8587a1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:30.524304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38972122-8616-4a5e-ae5c-de665b8587a1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:07:30.526229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-082e6e7d-9f20-41f8-bce8-d1da7b12f2af admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:30.526813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38972122-8616-4a5e-ae5c-de665b8587a1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:30.530405 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38972122-8616-4a5e-ae5c-de665b8587a1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:30.550066 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 897/1790] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:30.551601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 894/1791] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:07:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:07:30.554766 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e6e7d-9f20-41f8-bce8-d1da7b12f2af admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:30.556528 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38972122-8616-4a5e-ae5c-de665b8587a1 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:30.568435 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-082e6e7d-9f20-41f8-bce8-d1da7b12f2af admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:30.572374 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38972122-8616-4a5e-ae5c-de665b8587a1 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:30.573457 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53deded5-36a5-401d-bfc0-9fa2131d4283 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:30.574713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53deded5-36a5-401d-bfc0-9fa2131d4283 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:30.575389 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53deded5-36a5-401d-bfc0-9fa2131d4283 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:30.575776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53deded5-36a5-401d-bfc0-9fa2131d4283 admin admin] PATH_INFO: `/v3/projects/0e826af69dec41a19d213faa3b49ec22/users/1ce1a4983f9e4bdbb6c40cf36cd1de0f/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:30.581068 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53deded5-36a5-401d-bfc0-9fa2131d4283 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0e826af69dec41a19d213faa3b49ec22, user_id=1ce1a4983f9e4bdbb6c40cf36cd1de0f, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:30.582429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303d582-8316-497a-a0be-7371df29e463 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:30.583478 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303d582-8316-497a-a0be-7371df29e463 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:30.584139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303d582-8316-497a-a0be-7371df29e463 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:30.584548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53deded5-36a5-401d-bfc0-9fa2131d4283 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:30.584840 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a303d582-8316-497a-a0be-7371df29e463 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:30.586424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303d582-8316-497a-a0be-7371df29e463 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:30.589664 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a303d582-8316-497a-a0be-7371df29e463 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:30.597568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 898/1792] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:30 2022] PUT /identity/v3/projects/0e826af69dec41a19d213faa3b49ec22/users/1ce1a4983f9e4bdbb6c40cf36cd1de0f/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:30.607537 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce3679-b394-4d95-816d-da40bf6d0ea6 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:30.607841 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce3679-b394-4d95-816d-da40bf6d0ea6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:30.608114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcce3679-b394-4d95-816d-da40bf6d0ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:30.626348 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 895/1793] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:30 2022] POST /identity/v3/projects => generated 367 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:30.630839 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303d582-8316-497a-a0be-7371df29e463 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:30.646876 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-dcce3679-b394-4d95-816d-da40bf6d0ea6 None None] MFA Rules not processed for user `1ce1a4983f9e4bdbb6c40cf36cd1de0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:30.648335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dcce3679-b394-4d95-816d-da40bf6d0ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:30.726772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:30.744308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a303d582-8316-497a-a0be-7371df29e463 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:30.750982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54a278-b0aa-4c4a-b4cf-23a0002c1794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:30.751849 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54a278-b0aa-4c4a-b4cf-23a0002c1794 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:30.752143 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54a278-b0aa-4c4a-b4cf-23a0002c1794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:30.752426 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee54a278-b0aa-4c4a-b4cf-23a0002c1794 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:30.753135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54a278-b0aa-4c4a-b4cf-23a0002c1794 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:30.755323 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee54a278-b0aa-4c4a-b4cf-23a0002c1794 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:30.780301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 899/1794] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:30 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:30.793413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce3679-b394-4d95-816d-da40bf6d0ea6 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:30.796291 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dcce3679-b394-4d95-816d-da40bf6d0ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:30.808364 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 896/1795] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:30 2022] POST /identity/v3/users => generated 284 bytes in 179 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:30.812022 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54a278-b0aa-4c4a-b4cf-23a0002c1794 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:30.836048 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee54a278-b0aa-4c4a-b4cf-23a0002c1794 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:30.843161 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cab9f-19bb-4e6f-809e-1a3d14927fb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:30.844520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cab9f-19bb-4e6f-809e-1a3d14927fb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:30.845145 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cab9f-19bb-4e6f-809e-1a3d14927fb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:30.845614 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-482cab9f-19bb-4e6f-809e-1a3d14927fb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:30.846919 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482cab9f-19bb-4e6f-809e-1a3d14927fb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:30.849205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-482cab9f-19bb-4e6f-809e-1a3d14927fb8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:30.864903 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 897/1796] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:30.868605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cab9f-19bb-4e6f-809e-1a3d14927fb8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:30.890589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-482cab9f-19bb-4e6f-809e-1a3d14927fb8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:30.903690 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353adc53-3934-4c6d-84d1-d187561d0f23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:30.905126 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353adc53-3934-4c6d-84d1-d187561d0f23 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:30.905838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353adc53-3934-4c6d-84d1-d187561d0f23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:30.906531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-353adc53-3934-4c6d-84d1-d187561d0f23 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:30.908360 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353adc53-3934-4c6d-84d1-d187561d0f23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:30.910279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-353adc53-3934-4c6d-84d1-d187561d0f23 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:30.923618 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcce3679-b394-4d95-816d-da40bf6d0ea6 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:30.924656 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 898/1797] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:30 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:30.929575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18636f35-3a2a-4892-ba7c-b14432f1a62d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:30.929776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18636f35-3a2a-4892-ba7c-b14432f1a62d service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:30.930067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18636f35-3a2a-4892-ba7c-b14432f1a62d service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:30.930471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18636f35-3a2a-4892-ba7c-b14432f1a62d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:30.930792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353adc53-3934-4c6d-84d1-d187561d0f23 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:30.944823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-353adc53-3934-4c6d-84d1-d187561d0f23 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:30.950234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18636f35-3a2a-4892-ba7c-b14432f1a62d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:30.951686 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2ceda-69aa-4514-a761-f651ad46827b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:30.951814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18636f35-3a2a-4892-ba7c-b14432f1a62d service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:30.952933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2ceda-69aa-4514-a761-f651ad46827b admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:30.953692 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2ceda-69aa-4514-a761-f651ad46827b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:30.965002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd2ceda-69aa-4514-a761-f651ad46827b admin admin] PATH_INFO: `/v3/projects/1e04ac1072e3472092bca9f0c2a4d5df/users/6af63434c5314af89059e53ca2bb18d6/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:30.977233 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 900/1798] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:30 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 185 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:31.006869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2ceda-69aa-4514-a761-f651ad46827b admin admin] RBAC: Authorizing `identity:create_grant(project_id=1e04ac1072e3472092bca9f0c2a4d5df, user_id=6af63434c5314af89059e53ca2bb18d6, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:31.024251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd2ceda-69aa-4514-a761-f651ad46827b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:31.052796 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 899/1799] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:30 2022] PUT /identity/v3/projects/1e04ac1072e3472092bca9f0c2a4d5df/users/6af63434c5314af89059e53ca2bb18d6/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:31.063414 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c6b04-cb85-46a0-bb9f-6b9fb33c10f9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:31.065120 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c6b04-cb85-46a0-bb9f-6b9fb33c10f9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:31.065488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4c6b04-cb85-46a0-bb9f-6b9fb33c10f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:31.141644 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2ceda-69aa-4514-a761-f651ad46827b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:31.155844 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:31.165807 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-aa4c6b04-cb85-46a0-bb9f-6b9fb33c10f9 None None] MFA Rules not processed for user `6af63434c5314af89059e53ca2bb18d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:31.167113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-aa4c6b04-cb85-46a0-bb9f-6b9fb33c10f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:31.229940 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd2ceda-69aa-4514-a761-f651ad46827b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:31.234060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be5116-b9ba-4aa4-a6c0-2ba2b8bda93d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:31.236692 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be5116-b9ba-4aa4-a6c0-2ba2b8bda93d service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:31.237094 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be5116-b9ba-4aa4-a6c0-2ba2b8bda93d service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:31.237492 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10be5116-b9ba-4aa4-a6c0-2ba2b8bda93d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:31.250701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-10be5116-b9ba-4aa4-a6c0-2ba2b8bda93d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:31.286835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 901/1800] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:31 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:31.293526 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c6b04-cb85-46a0-bb9f-6b9fb33c10f9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:31.387471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be5116-b9ba-4aa4-a6c0-2ba2b8bda93d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:31.389229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10be5116-b9ba-4aa4-a6c0-2ba2b8bda93d service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:31.418838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 900/1801] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:31 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:07:31.431896 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4c6b04-cb85-46a0-bb9f-6b9fb33c10f9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:31.440477 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be5116-b9ba-4aa4-a6c0-2ba2b8bda93d service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:31.441938 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-10be5116-b9ba-4aa4-a6c0-2ba2b8bda93d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:31.447037 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f1c4f-8c6b-4b5c-bc38-c4b5dc3bf187 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:31.449281 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f1c4f-8c6b-4b5c-bc38-c4b5dc3bf187 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:31.449713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f1c4f-8c6b-4b5c-bc38-c4b5dc3bf187 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:31.450794 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-226f1c4f-8c6b-4b5c-bc38-c4b5dc3bf187 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:31.453800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f1c4f-8c6b-4b5c-bc38-c4b5dc3bf187 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:31.456100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-226f1c4f-8c6b-4b5c-bc38-c4b5dc3bf187 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:31.465632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10be5116-b9ba-4aa4-a6c0-2ba2b8bda93d service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:31.476397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d2703a-c1bf-43ce-9445-7419043197ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:31.478469 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2703a-c1bf-43ce-9445-7419043197ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:31.478590 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2703a-c1bf-43ce-9445-7419043197ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:31.478783 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d2703a-c1bf-43ce-9445-7419043197ef admin admin] PATH_INFO: `/v3/projects/85555e55da524204abeb8d782ddee1a9` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:31.481330 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d2703a-c1bf-43ce-9445-7419043197ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=85555e55da524204abeb8d782ddee1a9)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:31.483538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d2703a-c1bf-43ce-9445-7419043197ef admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:31.487595 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:31.514909 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 902/1802] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:31 2022] POST /identity/v3/projects => generated 367 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:31.516043 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f1c4f-8c6b-4b5c-bc38-c4b5dc3bf187 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:31.604709 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-47d2703a-c1bf-43ce-9445-7419043197ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '85555e55da524204abeb8d782ddee1a9', 'request_id': 'req-47d2703a-c1bf-43ce-9445-7419043197ef'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:31.616343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-47d2703a-c1bf-43ce-9445-7419043197ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '85555e55da524204abeb8d782ddee1a9', 'request_id': 'req-47d2703a-c1bf-43ce-9445-7419043197ef'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:31.625512 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-226f1c4f-8c6b-4b5c-bc38-c4b5dc3bf187 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:31.625945 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-47d2703a-c1bf-43ce-9445-7419043197ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '85555e55da524204abeb8d782ddee1a9', 'request_id': 'req-47d2703a-c1bf-43ce-9445-7419043197ef'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:31.644544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-47d2703a-c1bf-43ce-9445-7419043197ef admin admin] The token cache is being invalidate because project 85555e55da524204abeb8d782ddee1a9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:31.644844 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-47d2703a-c1bf-43ce-9445-7419043197ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:31.645691 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-47d2703a-c1bf-43ce-9445-7419043197ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:31.652051 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 901/1803] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:31 2022] DELETE /identity/v3/projects/85555e55da524204abeb8d782ddee1a9 => generated 0 bytes in 208 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:31.659651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03841cdd-8a69-48e1-a214-fcd9d886142e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:31.659953 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03841cdd-8a69-48e1-a214-fcd9d886142e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:31.660243 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03841cdd-8a69-48e1-a214-fcd9d886142e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:31.692130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-03841cdd-8a69-48e1-a214-fcd9d886142e None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:31.693255 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-03841cdd-8a69-48e1-a214-fcd9d886142e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:31.738067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b389510-1b9d-440d-9bc3-739e600dc48b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:31.739438 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b389510-1b9d-440d-9bc3-739e600dc48b admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:31.740300 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b389510-1b9d-440d-9bc3-739e600dc48b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:31.740786 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b389510-1b9d-440d-9bc3-739e600dc48b admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:31.741694 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b389510-1b9d-440d-9bc3-739e600dc48b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:31.745499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b389510-1b9d-440d-9bc3-739e600dc48b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:31.774134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:31.777478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 903/1804] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:31 2022] POST /identity/v3/users => generated 290 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:31.781532 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b389510-1b9d-440d-9bc3-739e600dc48b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:31.783888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3b389510-1b9d-440d-9bc3-739e600dc48b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:31.804923 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b389510-1b9d-440d-9bc3-739e600dc48b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:31.808682 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 902/1805] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:31.818463 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027e9d1-482e-4853-ab4e-be178b0350e8 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:31.819953 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027e9d1-482e-4853-ab4e-be178b0350e8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:31.820474 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5027e9d1-482e-4853-ab4e-be178b0350e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:31.826628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616099a4-1a6f-42da-a985-3217717d9b90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:31.828049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616099a4-1a6f-42da-a985-3217717d9b90 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:31.828563 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616099a4-1a6f-42da-a985-3217717d9b90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:31.829154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-616099a4-1a6f-42da-a985-3217717d9b90 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:31.833047 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616099a4-1a6f-42da-a985-3217717d9b90 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:31.836136 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-616099a4-1a6f-42da-a985-3217717d9b90 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:31.854371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 904/1806] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:07:31.855611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616099a4-1a6f-42da-a985-3217717d9b90 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:31.861227 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5027e9d1-482e-4853-ab4e-be178b0350e8 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:31.863138 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5027e9d1-482e-4853-ab4e-be178b0350e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:31.877100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-616099a4-1a6f-42da-a985-3217717d9b90 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:31.886370 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 903/1807] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:31.894095 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb0251-169d-4f4b-89f4-867e3b29d58c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:31.895387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb0251-169d-4f4b-89f4-867e3b29d58c admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:31.895725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb0251-169d-4f4b-89f4-867e3b29d58c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:31.895953 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7abb0251-169d-4f4b-89f4-867e3b29d58c admin admin] PATH_INFO: `/v3/projects/bd04a81f2c5d4906a17dfd2925de70e0/users/d9ff3c24b855407abf607f233fbf3ee8/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:31.897612 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027e9d1-482e-4853-ab4e-be178b0350e8 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:31.917140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb0251-169d-4f4b-89f4-867e3b29d58c admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd04a81f2c5d4906a17dfd2925de70e0, user_id=d9ff3c24b855407abf607f233fbf3ee8, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:31.919955 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5027e9d1-482e-4853-ab4e-be178b0350e8 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:31.920943 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7abb0251-169d-4f4b-89f4-867e3b29d58c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:31.932565 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952dbda-3470-4246-bf9c-938f786df30d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:31.933740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952dbda-3470-4246-bf9c-938f786df30d admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:31.934980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952dbda-3470-4246-bf9c-938f786df30d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:31.935399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e952dbda-3470-4246-bf9c-938f786df30d admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:31.937561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952dbda-3470-4246-bf9c-938f786df30d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:07:31.939042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952dbda-3470-4246-bf9c-938f786df30d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:31.955025 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e952dbda-3470-4246-bf9c-938f786df30d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:31.964237 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 905/1808] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:31 2022] PUT /identity/v3/projects/bd04a81f2c5d4906a17dfd2925de70e0/users/d9ff3c24b855407abf607f233fbf3ee8/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:31.965791 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb0251-169d-4f4b-89f4-867e3b29d58c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:31.996232 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 904/1809] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:07:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:07:32.001563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952dbda-3470-4246-bf9c-938f786df30d admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:32.013917 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e952dbda-3470-4246-bf9c-938f786df30d admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:32.105417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:32.110721 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a172f-339f-4c08-82c2-99c50c8e26cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:32.111413 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a172f-339f-4c08-82c2-99c50c8e26cf admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:32.111701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a172f-339f-4c08-82c2-99c50c8e26cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:32.111979 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f1a172f-339f-4c08-82c2-99c50c8e26cf admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:32.112692 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a172f-339f-4c08-82c2-99c50c8e26cf admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:32.116553 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7abb0251-169d-4f4b-89f4-867e3b29d58c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:32.118525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f1a172f-339f-4c08-82c2-99c50c8e26cf admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:32.126170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcecd8a-4c43-487a-96fb-e1ec6ced5d13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:32.126906 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcecd8a-4c43-487a-96fb-e1ec6ced5d13 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:32.127193 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcecd8a-4c43-487a-96fb-e1ec6ced5d13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:32.127475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bcecd8a-4c43-487a-96fb-e1ec6ced5d13 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:32.128227 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcecd8a-4c43-487a-96fb-e1ec6ced5d13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:32.129854 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bcecd8a-4c43-487a-96fb-e1ec6ced5d13 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:32.142349 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 906/1810] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:31 2022] GET /identity/v3/roles => generated 1420 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:32.146994 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcecd8a-4c43-487a-96fb-e1ec6ced5d13 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:32.159479 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 905/1811] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:32 2022] POST /identity/v3/projects => generated 381 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:32.162961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bcecd8a-4c43-487a-96fb-e1ec6ced5d13 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:32.163895 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a172f-339f-4c08-82c2-99c50c8e26cf admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:32.247002 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88e56227-2142-40be-86e9-9cfb7f95cd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:32.248212 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e56227-2142-40be-86e9-9cfb7f95cd90 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:32.248505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e56227-2142-40be-86e9-9cfb7f95cd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:32.248790 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88e56227-2142-40be-86e9-9cfb7f95cd90 admin admin] PATH_INFO: `/v3/projects/bd04a81f2c5d4906a17dfd2925de70e0/users/d9ff3c24b855407abf607f233fbf3ee8/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:32.249740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f1a172f-339f-4c08-82c2-99c50c8e26cf admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:32.251724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e56227-2142-40be-86e9-9cfb7f95cd90 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd04a81f2c5d4906a17dfd2925de70e0, user_id=d9ff3c24b855407abf607f233fbf3ee8, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:32.254525 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88e56227-2142-40be-86e9-9cfb7f95cd90 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:32.255908 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d15809-8cd9-4f2c-8b0d-f70d422b1d8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:32.256597 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d15809-8cd9-4f2c-8b0d-f70d422b1d8b admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:32.257066 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d15809-8cd9-4f2c-8b0d-f70d422b1d8b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:32.257353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d15809-8cd9-4f2c-8b0d-f70d422b1d8b admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:32.258538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d15809-8cd9-4f2c-8b0d-f70d422b1d8b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:32.261817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d15809-8cd9-4f2c-8b0d-f70d422b1d8b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:32.284293 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 907/1812] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:32 2022] PUT /identity/v3/projects/bd04a81f2c5d4906a17dfd2925de70e0/users/d9ff3c24b855407abf607f233fbf3ee8/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:32.284293 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa05a3a-4ca1-4445-a675-a6683736a1d2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:32.284293 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa05a3a-4ca1-4445-a675-a6683736a1d2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:32.284293 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa05a3a-4ca1-4445-a675-a6683736a1d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:32.304720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 906/1813] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:32 2022] POST /identity/v3/users => generated 291 bytes in 142 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:32.308773 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d15809-8cd9-4f2c-8b0d-f70d422b1d8b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:32.317117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-efa05a3a-4ca1-4445-a675-a6683736a1d2 None None] MFA Rules not processed for user `d9ff3c24b855407abf607f233fbf3ee8`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:32.318798 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-efa05a3a-4ca1-4445-a675-a6683736a1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:32.327329 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d15809-8cd9-4f2c-8b0d-f70d422b1d8b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:32.387488 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:32.413011 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 908/1814] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:32 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:32.413653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad34bd-1289-4dbf-bc38-7c14f16b09a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:32.414376 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad34bd-1289-4dbf-bc38-7c14f16b09a8 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:32.414670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad34bd-1289-4dbf-bc38-7c14f16b09a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:32.414954 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fad34bd-1289-4dbf-bc38-7c14f16b09a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:32.415704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad34bd-1289-4dbf-bc38-7c14f16b09a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:32.419238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fad34bd-1289-4dbf-bc38-7c14f16b09a8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:32.425114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa05a3a-4ca1-4445-a675-a6683736a1d2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:32.426435 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-efa05a3a-4ca1-4445-a675-a6683736a1d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:32.442466 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 907/1815] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:32.446394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad34bd-1289-4dbf-bc38-7c14f16b09a8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:32.461827 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fad34bd-1289-4dbf-bc38-7c14f16b09a8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:32.466843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a9042-85c7-4a63-bfdb-13c545f5d449 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:32.468072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a9042-85c7-4a63-bfdb-13c545f5d449 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:32.468367 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a9042-85c7-4a63-bfdb-13c545f5d449 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:32.468683 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997a9042-85c7-4a63-bfdb-13c545f5d449 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:32.469434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a9042-85c7-4a63-bfdb-13c545f5d449 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:32.471096 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997a9042-85c7-4a63-bfdb-13c545f5d449 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:32.486545 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 908/1816] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:32.487284 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a9042-85c7-4a63-bfdb-13c545f5d449 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:32.499598 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997a9042-85c7-4a63-bfdb-13c545f5d449 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:32.504796 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea237aed-825b-416d-ad56-d27a883a4a41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:32.505766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea237aed-825b-416d-ad56-d27a883a4a41 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:32.506140 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea237aed-825b-416d-ad56-d27a883a4a41 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:32.506589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea237aed-825b-416d-ad56-d27a883a4a41 admin admin] PATH_INFO: `/v3/projects/9496150416714a978d8dddab24046533/users/066ed239ec4444cf80d7638951c85438/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:32.517229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa05a3a-4ca1-4445-a675-a6683736a1d2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:32.521023 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fed4eff-a81a-463d-a1c2-6c5d3431075d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:32.521689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed4eff-a81a-463d-a1c2-6c5d3431075d service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:32.521971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed4eff-a81a-463d-a1c2-6c5d3431075d service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:32.522275 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fed4eff-a81a-463d-a1c2-6c5d3431075d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:32.532695 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea237aed-825b-416d-ad56-d27a883a4a41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9496150416714a978d8dddab24046533, user_id=066ed239ec4444cf80d7638951c85438, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:32.533517 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed4eff-a81a-463d-a1c2-6c5d3431075d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:32.535000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fed4eff-a81a-463d-a1c2-6c5d3431075d service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:32.535447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea237aed-825b-416d-ad56-d27a883a4a41 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:32.546116 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 909/1817] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:32 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:07:32.549364 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 909/1818] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:32 2022] PUT /identity/v3/projects/9496150416714a978d8dddab24046533/users/066ed239ec4444cf80d7638951c85438/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:32.561538 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bdb93-3645-4b31-9a5c-75dd6662ed47 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:32.561838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bdb93-3645-4b31-9a5c-75dd6662ed47 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:32.562122 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3bdb93-3645-4b31-9a5c-75dd6662ed47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:32.592468 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5f3bdb93-3645-4b31-9a5c-75dd6662ed47 None None] MFA Rules not processed for user `066ed239ec4444cf80d7638951c85438`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:32.594338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5f3bdb93-3645-4b31-9a5c-75dd6662ed47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:32.598811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:32.672788 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 910/1819] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:32 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:32.677658 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea237aed-825b-416d-ad56-d27a883a4a41 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:32.746284 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea237aed-825b-416d-ad56-d27a883a4a41 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:32.751532 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bdd882-d881-4d8b-b0d3-9962785843fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:32.752194 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdd882-d881-4d8b-b0d3-9962785843fe admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:32.752485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdd882-d881-4d8b-b0d3-9962785843fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:32.752762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68bdd882-d881-4d8b-b0d3-9962785843fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:32.753492 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bdd882-d881-4d8b-b0d3-9962785843fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:32.755469 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68bdd882-d881-4d8b-b0d3-9962785843fe admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:32.783044 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 910/1820] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:32 2022] POST /identity/v3/projects => generated 381 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:32.787087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3bdb93-3645-4b31-9a5c-75dd6662ed47 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:32.798788 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3bdb93-3645-4b31-9a5c-75dd6662ed47 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:32.865173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1628aa5-7392-451b-9067-97b77c9d0122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:32.865869 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1628aa5-7392-451b-9067-97b77c9d0122 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:32.866152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1628aa5-7392-451b-9067-97b77c9d0122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:32.866465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1628aa5-7392-451b-9067-97b77c9d0122 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:32.867179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1628aa5-7392-451b-9067-97b77c9d0122 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:32.869112 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1628aa5-7392-451b-9067-97b77c9d0122 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:32.893896 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 911/1821] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:32 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:32.898329 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bdd882-d881-4d8b-b0d3-9962785843fe admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:32.916605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68bdd882-d881-4d8b-b0d3-9962785843fe admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:32.927616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978fe30-e777-42e9-8e1d-c98acd388b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:32.928422 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978fe30-e777-42e9-8e1d-c98acd388b73 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:32.928831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978fe30-e777-42e9-8e1d-c98acd388b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:32.929119 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d978fe30-e777-42e9-8e1d-c98acd388b73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:32.930489 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978fe30-e777-42e9-8e1d-c98acd388b73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:32.932789 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d978fe30-e777-42e9-8e1d-c98acd388b73 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:32.958339 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 911/1822] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:32 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:32.962493 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1628aa5-7392-451b-9067-97b77c9d0122 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:32.975425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1628aa5-7392-451b-9067-97b77c9d0122 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:32.982449 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ed6dc-fd44-4482-b135-5459138db533 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:32.986048 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ed6dc-fd44-4482-b135-5459138db533 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:32.986498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ed6dc-fd44-4482-b135-5459138db533 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:32.987361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3ed6dc-fd44-4482-b135-5459138db533 admin admin] PATH_INFO: `/v3/projects/600f27b5d2f94f839ee3cbfd06b69e74/users/c199786bfafa4c6ab5a3534759f85141/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:33.009072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ed6dc-fd44-4482-b135-5459138db533 admin admin] RBAC: Authorizing `identity:create_grant(project_id=600f27b5d2f94f839ee3cbfd06b69e74, user_id=c199786bfafa4c6ab5a3534759f85141, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:33.012936 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3ed6dc-fd44-4482-b135-5459138db533 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:33.044128 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 912/1823] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:32 2022] PUT /identity/v3/projects/600f27b5d2f94f839ee3cbfd06b69e74/users/c199786bfafa4c6ab5a3534759f85141/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:33.045997 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978fe30-e777-42e9-8e1d-c98acd388b73 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:33.062077 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d978fe30-e777-42e9-8e1d-c98acd388b73 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:33.205148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-267d1418-c7a5-441c-bac0-51e77d4e07cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:33.212902 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d1418-c7a5-441c-bac0-51e77d4e07cb admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:33.212902 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d1418-c7a5-441c-bac0-51e77d4e07cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:33.212902 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-267d1418-c7a5-441c-bac0-51e77d4e07cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:33.212902 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d1418-c7a5-441c-bac0-51e77d4e07cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:33.216104 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-267d1418-c7a5-441c-bac0-51e77d4e07cb admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:33.237331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 912/1824] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:33 2022] GET /identity/v3/roles => generated 1420 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:33.242448 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ed6dc-fd44-4482-b135-5459138db533 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:33.260141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3ed6dc-fd44-4482-b135-5459138db533 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:33.274840 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bdd30-4231-4051-9231-31aa4a0ee7f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:33.275678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bdd30-4231-4051-9231-31aa4a0ee7f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:33.275961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bdd30-4231-4051-9231-31aa4a0ee7f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:33.276234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0bdd30-4231-4051-9231-31aa4a0ee7f8 admin admin] PATH_INFO: `/v3/projects/600f27b5d2f94f839ee3cbfd06b69e74/users/c199786bfafa4c6ab5a3534759f85141/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:33.278679 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bdd30-4231-4051-9231-31aa4a0ee7f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=600f27b5d2f94f839ee3cbfd06b69e74, user_id=c199786bfafa4c6ab5a3534759f85141, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:33.284907 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0bdd30-4231-4051-9231-31aa4a0ee7f8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:33.327002 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 913/1825] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:33 2022] PUT /identity/v3/projects/600f27b5d2f94f839ee3cbfd06b69e74/users/c199786bfafa4c6ab5a3534759f85141/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:33.339386 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b8b76-42d4-46bb-88c0-d78c2113dc17 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:33.339686 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b8b76-42d4-46bb-88c0-d78c2113dc17 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:33.340035 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-081b8b76-42d4-46bb-88c0-d78c2113dc17 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:33.404557 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:33.449245 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-081b8b76-42d4-46bb-88c0-d78c2113dc17 None None] MFA Rules not processed for user `c199786bfafa4c6ab5a3534759f85141`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:33.451160 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-081b8b76-42d4-46bb-88c0-d78c2113dc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:33.563454 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 913/1826] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:33.576165 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bdd30-4231-4051-9231-31aa4a0ee7f8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:33.581781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3b0bdd30-4231-4051-9231-31aa4a0ee7f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:33.658648 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0bdd30-4231-4051-9231-31aa4a0ee7f8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:33.675270 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d7a44-852a-4e90-b5d8-6961b8aa2f34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:33.675955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d7a44-852a-4e90-b5d8-6961b8aa2f34 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:33.676241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d7a44-852a-4e90-b5d8-6961b8aa2f34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:33.676521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96d7a44-852a-4e90-b5d8-6961b8aa2f34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:33.689926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d7a44-852a-4e90-b5d8-6961b8aa2f34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:33.691387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96d7a44-852a-4e90-b5d8-6961b8aa2f34 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:33.698824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:33.722631 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 914/1827] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:33 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:34.485356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b8b76-42d4-46bb-88c0-d78c2113dc17 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:34.488527 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-081b8b76-42d4-46bb-88c0-d78c2113dc17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:34.538449 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d7a44-852a-4e90-b5d8-6961b8aa2f34 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:34.682991 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96d7a44-852a-4e90-b5d8-6961b8aa2f34 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:34.691792 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d69658-e1e8-426a-beb9-23e2a15c8e56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:34.693329 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d69658-e1e8-426a-beb9-23e2a15c8e56 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:34.693857 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d69658-e1e8-426a-beb9-23e2a15c8e56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:34.694338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75d69658-e1e8-426a-beb9-23e2a15c8e56 admin admin] PATH_INFO: `/v3/users/066ed239ec4444cf80d7638951c85438` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:34.696879 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d69658-e1e8-426a-beb9-23e2a15c8e56 admin admin] RBAC: Authorizing `identity:delete_user(user_id=066ed239ec4444cf80d7638951c85438)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:34.700000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75d69658-e1e8-426a-beb9-23e2a15c8e56 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:34.737748 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-081b8b76-42d4-46bb-88c0-d78c2113dc17 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:34.744230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6941c6-c015-418c-a9c0-a84e90511603 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:34.745671 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6941c6-c015-418c-a9c0-a84e90511603 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:34.745671 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6941c6-c015-418c-a9c0-a84e90511603 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:34.750924 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6941c6-c015-418c-a9c0-a84e90511603 admin admin] PATH_INFO: `/v3/users/17285d4bb61245b19331aa0c915a5e4b` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:34.752826 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6941c6-c015-418c-a9c0-a84e90511603 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17285d4bb61245b19331aa0c915a5e4b)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:34.754963 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6941c6-c015-418c-a9c0-a84e90511603 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:34.782773 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-75d69658-e1e8-426a-beb9-23e2a15c8e56 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '066ed239ec4444cf80d7638951c85438'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:34.783367 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-75d69658-e1e8-426a-beb9-23e2a15c8e56 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '066ed239ec4444cf80d7638951c85438'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:34.801360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-75d69658-e1e8-426a-beb9-23e2a15c8e56 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '066ed239ec4444cf80d7638951c85438'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:34.802334 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-75d69658-e1e8-426a-beb9-23e2a15c8e56 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '066ed239ec4444cf80d7638951c85438'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:34.842906 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-75d69658-e1e8-426a-beb9-23e2a15c8e56 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '066ed239ec4444cf80d7638951c85438'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:34.860160 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0d6941c6-c015-418c-a9c0-a84e90511603 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17285d4bb61245b19331aa0c915a5e4b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:34.879419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 915/1828] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:34 2022] DELETE /identity/v3/users/066ed239ec4444cf80d7638951c85438 => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:34.879419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d69658-e1e8-426a-beb9-23e2a15c8e56 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:34.884575 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0d6941c6-c015-418c-a9c0-a84e90511603 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17285d4bb61245b19331aa0c915a5e4b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:34.885283 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-75d69658-e1e8-426a-beb9-23e2a15c8e56 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:34.908398 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0d6941c6-c015-418c-a9c0-a84e90511603 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17285d4bb61245b19331aa0c915a5e4b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:34.909241 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0d6941c6-c015-418c-a9c0-a84e90511603 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17285d4bb61245b19331aa0c915a5e4b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:34.926569 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0d6941c6-c015-418c-a9c0-a84e90511603 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17285d4bb61245b19331aa0c915a5e4b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:34.939444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 914/1829] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:34 2022] DELETE /identity/v3/users/17285d4bb61245b19331aa0c915a5e4b => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:34.939444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a31adc-7464-4e6a-8fca-a8afc3eda06b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:34.939444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a31adc-7464-4e6a-8fca-a8afc3eda06b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:34.940092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a31adc-7464-4e6a-8fca-a8afc3eda06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:34.963632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-74a31adc-7464-4e6a-8fca-a8afc3eda06b None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:34.964947 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-74a31adc-7464-4e6a-8fca-a8afc3eda06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:34.977713 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:35.036811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75d69658-e1e8-426a-beb9-23e2a15c8e56 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:35.084408 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:35.153076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 915/1830] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:35.157868 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d70ff-e09b-4f7d-a655-790895c4f17d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:35.158165 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d70ff-e09b-4f7d-a655-790895c4f17d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:35.158484 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3d70ff-e09b-4f7d-a655-790895c4f17d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:35.178345 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-fa3d70ff-e09b-4f7d-a655-790895c4f17d None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:35.179601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-fa3d70ff-e09b-4f7d-a655-790895c4f17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:35.188184 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 916/1831] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:35.189154 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d70ff-e09b-4f7d-a655-790895c4f17d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:35.192652 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-fa3d70ff-e09b-4f7d-a655-790895c4f17d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:35.193901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e6a02-123b-47ea-9e86-7c9e74b81a08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:35.196083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e6a02-123b-47ea-9e86-7c9e74b81a08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:35.197264 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e6a02-123b-47ea-9e86-7c9e74b81a08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:35.197832 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-555e6a02-123b-47ea-9e86-7c9e74b81a08 admin admin] PATH_INFO: `/v3/users/6af63434c5314af89059e53ca2bb18d6` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:35.201416 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e6a02-123b-47ea-9e86-7c9e74b81a08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6af63434c5314af89059e53ca2bb18d6)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:35.205847 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-555e6a02-123b-47ea-9e86-7c9e74b81a08 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:35.288202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-555e6a02-123b-47ea-9e86-7c9e74b81a08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6af63434c5314af89059e53ca2bb18d6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:35.290205 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-555e6a02-123b-47ea-9e86-7c9e74b81a08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6af63434c5314af89059e53ca2bb18d6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:35.307866 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3d70ff-e09b-4f7d-a655-790895c4f17d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:35.308889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-555e6a02-123b-47ea-9e86-7c9e74b81a08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6af63434c5314af89059e53ca2bb18d6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:35.309584 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-555e6a02-123b-47ea-9e86-7c9e74b81a08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6af63434c5314af89059e53ca2bb18d6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:35.317285 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ae05c91-3d70-48ec-a338-fe2b72871354 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:35.319431 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae05c91-3d70-48ec-a338-fe2b72871354 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:35.320613 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae05c91-3d70-48ec-a338-fe2b72871354 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:35.321161 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ae05c91-3d70-48ec-a338-fe2b72871354 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:35.324977 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8ae05c91-3d70-48ec-a338-fe2b72871354 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:35.327194 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-555e6a02-123b-47ea-9e86-7c9e74b81a08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6af63434c5314af89059e53ca2bb18d6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:35.344559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 916/1832] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:34 2022] DELETE /identity/v3/users/6af63434c5314af89059e53ca2bb18d6 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:35.345799 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e6a02-123b-47ea-9e86-7c9e74b81a08 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:35.346510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae05c91-3d70-48ec-a338-fe2b72871354 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:35.348337 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ae05c91-3d70-48ec-a338-fe2b72871354 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:35.348620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-555e6a02-123b-47ea-9e86-7c9e74b81a08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:35.400233 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 917/1833] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:35.405259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22064597-d735-466b-8590-e3e550b4cb53 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:35.407209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22064597-d735-466b-8590-e3e550b4cb53 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:35.407636 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22064597-d735-466b-8590-e3e550b4cb53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:35.447482 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-555e6a02-123b-47ea-9e86-7c9e74b81a08 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:35.452475 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-22064597-d735-466b-8590-e3e550b4cb53 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:35.454146 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa582c5f-6808-469f-ab03-657d5eb4abc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:35.454709 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-22064597-d735-466b-8590-e3e550b4cb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:35.455060 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa582c5f-6808-469f-ab03-657d5eb4abc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:35.455527 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa582c5f-6808-469f-ab03-657d5eb4abc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:35.455845 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa582c5f-6808-469f-ab03-657d5eb4abc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:35.458516 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-aa582c5f-6808-469f-ab03-657d5eb4abc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:35.582549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:35.588871 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 918/1834] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:35.594345 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa582c5f-6808-469f-ab03-657d5eb4abc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:35.596094 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa582c5f-6808-469f-ab03-657d5eb4abc2 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:35.608324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 917/1835] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:35.610381 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22064597-d735-466b-8590-e3e550b4cb53 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:35.612316 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-22064597-d735-466b-8590-e3e550b4cb53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:35.625334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22064597-d735-466b-8590-e3e550b4cb53 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:35.630232 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e65d4a2a-308d-47bb-9aa1-070229fb9689 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:35.631864 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d4a2a-308d-47bb-9aa1-070229fb9689 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:35.632337 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d4a2a-308d-47bb-9aa1-070229fb9689 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:35.632879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e65d4a2a-308d-47bb-9aa1-070229fb9689 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:35.644268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d4a2a-308d-47bb-9aa1-070229fb9689 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:35.647499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e65d4a2a-308d-47bb-9aa1-070229fb9689 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:35.659652 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 919/1836] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:07:36.448565 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec398d6-5f54-47cb-b635-97405c74a217 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:36.448873 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec398d6-5f54-47cb-b635-97405c74a217 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:36.449156 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bec398d6-5f54-47cb-b635-97405c74a217 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:36.450601 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 918/1837] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 11 14:07:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:07:36.460787 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3866871-8f66-4412-96cb-b64d1e646ad9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:36.461083 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3866871-8f66-4412-96cb-b64d1e646ad9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:36.461372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3866871-8f66-4412-96cb-b64d1e646ad9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:36.490699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a3866871-8f66-4412-96cb-b64d1e646ad9 None None] MFA Rules not processed for user `ef5dac3f311e43809a46995bab56e780`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:36.491949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a3866871-8f66-4412-96cb-b64d1e646ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:36.560987 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:36.613537 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 920/1838] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 11 14:07:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:36.977643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec398d6-5f54-47cb-b635-97405c74a217 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:36.981071 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bec398d6-5f54-47cb-b635-97405c74a217 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:37.030020 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bec398d6-5f54-47cb-b635-97405c74a217 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:37.059932 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed40cb4-3108-4a66-811c-63056b0950ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:37.066994 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed40cb4-3108-4a66-811c-63056b0950ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:37.068101 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed40cb4-3108-4a66-811c-63056b0950ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:37.068715 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ed40cb4-3108-4a66-811c-63056b0950ee admin admin] PATH_INFO: `/v3/projects/9496150416714a978d8dddab24046533` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:37.073393 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed40cb4-3108-4a66-811c-63056b0950ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=9496150416714a978d8dddab24046533)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:37.079229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ed40cb4-3108-4a66-811c-63056b0950ee admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:37.153321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4ed40cb4-3108-4a66-811c-63056b0950ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9496150416714a978d8dddab24046533', 'request_id': 'req-4ed40cb4-3108-4a66-811c-63056b0950ee'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.160161 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4ed40cb4-3108-4a66-811c-63056b0950ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9496150416714a978d8dddab24046533', 'request_id': 'req-4ed40cb4-3108-4a66-811c-63056b0950ee'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.169602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4ed40cb4-3108-4a66-811c-63056b0950ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9496150416714a978d8dddab24046533', 'request_id': 'req-4ed40cb4-3108-4a66-811c-63056b0950ee'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.184830 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4ed40cb4-3108-4a66-811c-63056b0950ee admin admin] The token cache is being invalidate because project 9496150416714a978d8dddab24046533 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:37.185123 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4ed40cb4-3108-4a66-811c-63056b0950ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.185960 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4ed40cb4-3108-4a66-811c-63056b0950ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.191433 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 919/1839] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:36 2022] DELETE /identity/v3/projects/9496150416714a978d8dddab24046533 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:37.192797 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3866871-8f66-4412-96cb-b64d1e646ad9 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:37.205394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3866871-8f66-4412-96cb-b64d1e646ad9 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:37.282292 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5440c8-f992-4ae9-bf95-c1d3986e031f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:37.283303 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5440c8-f992-4ae9-bf95-c1d3986e031f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:37.283626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5440c8-f992-4ae9-bf95-c1d3986e031f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:37.283874 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5440c8-f992-4ae9-bf95-c1d3986e031f admin admin] PATH_INFO: `/v3/users/c199786bfafa4c6ab5a3534759f85141` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:37.285325 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5440c8-f992-4ae9-bf95-c1d3986e031f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c199786bfafa4c6ab5a3534759f85141)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:37.288331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5440c8-f992-4ae9-bf95-c1d3986e031f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:37.314332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed40cb4-3108-4a66-811c-63056b0950ee admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:37.316350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4ed40cb4-3108-4a66-811c-63056b0950ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:37.375910 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ed40cb4-3108-4a66-811c-63056b0950ee admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:37.380351 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ac5440c8-f992-4ae9-bf95-c1d3986e031f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c199786bfafa4c6ab5a3534759f85141'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.384658 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248f4c3-8e0e-4d8e-9bf4-2ca5d4c5a309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:37.385801 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248f4c3-8e0e-4d8e-9bf4-2ca5d4c5a309 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:37.386175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248f4c3-8e0e-4d8e-9bf4-2ca5d4c5a309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:37.386581 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3248f4c3-8e0e-4d8e-9bf4-2ca5d4c5a309 admin admin] PATH_INFO: `/v3/projects/cad3ea44ef7f421291cedcde0bf2d468` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:37.388813 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248f4c3-8e0e-4d8e-9bf4-2ca5d4c5a309 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cad3ea44ef7f421291cedcde0bf2d468)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:37.391608 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3248f4c3-8e0e-4d8e-9bf4-2ca5d4c5a309 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:37.400530 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ac5440c8-f992-4ae9-bf95-c1d3986e031f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c199786bfafa4c6ab5a3534759f85141'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.410485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ac5440c8-f992-4ae9-bf95-c1d3986e031f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c199786bfafa4c6ab5a3534759f85141'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.411429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ac5440c8-f992-4ae9-bf95-c1d3986e031f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c199786bfafa4c6ab5a3534759f85141'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.422672 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ac5440c8-f992-4ae9-bf95-c1d3986e031f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c199786bfafa4c6ab5a3534759f85141'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.430006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 921/1840] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:37 2022] DELETE /identity/v3/users/c199786bfafa4c6ab5a3534759f85141 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:37.431045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5440c8-f992-4ae9-bf95-c1d3986e031f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:37.432829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ac5440c8-f992-4ae9-bf95-c1d3986e031f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:37.455025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3248f4c3-8e0e-4d8e-9bf4-2ca5d4c5a309 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cad3ea44ef7f421291cedcde0bf2d468', 'request_id': 'req-3248f4c3-8e0e-4d8e-9bf4-2ca5d4c5a309'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.461665 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3248f4c3-8e0e-4d8e-9bf4-2ca5d4c5a309 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cad3ea44ef7f421291cedcde0bf2d468', 'request_id': 'req-3248f4c3-8e0e-4d8e-9bf4-2ca5d4c5a309'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.475405 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3248f4c3-8e0e-4d8e-9bf4-2ca5d4c5a309 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cad3ea44ef7f421291cedcde0bf2d468', 'request_id': 'req-3248f4c3-8e0e-4d8e-9bf4-2ca5d4c5a309'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.491768 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3248f4c3-8e0e-4d8e-9bf4-2ca5d4c5a309 admin admin] The token cache is being invalidate because project cad3ea44ef7f421291cedcde0bf2d468 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:37.492245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3248f4c3-8e0e-4d8e-9bf4-2ca5d4c5a309 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.493279 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3248f4c3-8e0e-4d8e-9bf4-2ca5d4c5a309 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.498005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 920/1841] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:37 2022] DELETE /identity/v3/projects/cad3ea44ef7f421291cedcde0bf2d468 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:37.498005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248f4c3-8e0e-4d8e-9bf4-2ca5d4c5a309 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:37.498005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3248f4c3-8e0e-4d8e-9bf4-2ca5d4c5a309 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:37.520543 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:37.538223 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5440c8-f992-4ae9-bf95-c1d3986e031f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:37.600176 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:37.607148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf1ef1-7107-4360-a14d-7bff39b0fd1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:37.608156 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf1ef1-7107-4360-a14d-7bff39b0fd1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:37.608490 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf1ef1-7107-4360-a14d-7bff39b0fd1a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:37.608769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbf1ef1-7107-4360-a14d-7bff39b0fd1a admin admin] PATH_INFO: `/v3/projects/1e04ac1072e3472092bca9f0c2a4d5df` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:37.610022 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf1ef1-7107-4360-a14d-7bff39b0fd1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1e04ac1072e3472092bca9f0c2a4d5df)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:37.612786 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbf1ef1-7107-4360-a14d-7bff39b0fd1a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:37.615919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3248f4c3-8e0e-4d8e-9bf4-2ca5d4c5a309 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:37.623531 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193437f-d5ea-4168-8efa-ce0397156ec6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:37.624396 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193437f-d5ea-4168-8efa-ce0397156ec6 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:37.624870 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193437f-d5ea-4168-8efa-ce0397156ec6 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:37.625246 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d193437f-d5ea-4168-8efa-ce0397156ec6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:37.627252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d193437f-d5ea-4168-8efa-ce0397156ec6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:37.642146 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193437f-d5ea-4168-8efa-ce0397156ec6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:37.643911 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d193437f-d5ea-4168-8efa-ce0397156ec6 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:37.658655 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 921/1842] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:37.659519 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193437f-d5ea-4168-8efa-ce0397156ec6 service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:37.660893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d193437f-d5ea-4168-8efa-ce0397156ec6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:37.680327 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d193437f-d5ea-4168-8efa-ce0397156ec6 service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:37.684931 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bdbf1ef1-7107-4360-a14d-7bff39b0fd1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1e04ac1072e3472092bca9f0c2a4d5df', 'request_id': 'req-bdbf1ef1-7107-4360-a14d-7bff39b0fd1a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.696440 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bdbf1ef1-7107-4360-a14d-7bff39b0fd1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1e04ac1072e3472092bca9f0c2a4d5df', 'request_id': 'req-bdbf1ef1-7107-4360-a14d-7bff39b0fd1a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.708171 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bdbf1ef1-7107-4360-a14d-7bff39b0fd1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1e04ac1072e3472092bca9f0c2a4d5df', 'request_id': 'req-bdbf1ef1-7107-4360-a14d-7bff39b0fd1a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.725632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bdbf1ef1-7107-4360-a14d-7bff39b0fd1a admin admin] The token cache is being invalidate because project 1e04ac1072e3472092bca9f0c2a4d5df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:37.726046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bdbf1ef1-7107-4360-a14d-7bff39b0fd1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.726697 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-bdbf1ef1-7107-4360-a14d-7bff39b0fd1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.729281 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 922/1843] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:37 2022] DELETE /identity/v3/projects/1e04ac1072e3472092bca9f0c2a4d5df => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:37.732438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf1ef1-7107-4360-a14d-7bff39b0fd1a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:37.733949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bdbf1ef1-7107-4360-a14d-7bff39b0fd1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:37.756839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824762c-f815-4a3a-b023-8a10406f6f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:37.757952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824762c-f815-4a3a-b023-8a10406f6f1f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:37.758099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824762c-f815-4a3a-b023-8a10406f6f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:37.758280 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5824762c-f815-4a3a-b023-8a10406f6f1f admin admin] PATH_INFO: `/v3/users/1ce1a4983f9e4bdbb6c40cf36cd1de0f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:37.759720 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824762c-f815-4a3a-b023-8a10406f6f1f admin admin] RBAC: Authorizing `identity:delete_user(user_id=1ce1a4983f9e4bdbb6c40cf36cd1de0f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:37.761649 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5824762c-f815-4a3a-b023-8a10406f6f1f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:37.810851 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbf1ef1-7107-4360-a14d-7bff39b0fd1a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:37.817662 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a566438b-ca75-401d-a5ce-b9c9e15b9af7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:37.820704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566438b-ca75-401d-a5ce-b9c9e15b9af7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:37.820956 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566438b-ca75-401d-a5ce-b9c9e15b9af7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:37.821153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a566438b-ca75-401d-a5ce-b9c9e15b9af7 admin admin] PATH_INFO: `/v3/users/d9ff3c24b855407abf607f233fbf3ee8` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:37.821701 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5824762c-f815-4a3a-b023-8a10406f6f1f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1ce1a4983f9e4bdbb6c40cf36cd1de0f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.822656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5824762c-f815-4a3a-b023-8a10406f6f1f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1ce1a4983f9e4bdbb6c40cf36cd1de0f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.823132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566438b-ca75-401d-a5ce-b9c9e15b9af7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9ff3c24b855407abf607f233fbf3ee8)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:37.826573 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a566438b-ca75-401d-a5ce-b9c9e15b9af7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:37.833725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5824762c-f815-4a3a-b023-8a10406f6f1f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1ce1a4983f9e4bdbb6c40cf36cd1de0f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.834396 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5824762c-f815-4a3a-b023-8a10406f6f1f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1ce1a4983f9e4bdbb6c40cf36cd1de0f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.847648 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5824762c-f815-4a3a-b023-8a10406f6f1f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1ce1a4983f9e4bdbb6c40cf36cd1de0f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.862819 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 922/1844] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:37 2022] DELETE /identity/v3/users/1ce1a4983f9e4bdbb6c40cf36cd1de0f => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:37.865833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824762c-f815-4a3a-b023-8a10406f6f1f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:37.865833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5824762c-f815-4a3a-b023-8a10406f6f1f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:37.871556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:37.899352 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a566438b-ca75-401d-a5ce-b9c9e15b9af7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9ff3c24b855407abf607f233fbf3ee8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.918014 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a566438b-ca75-401d-a5ce-b9c9e15b9af7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9ff3c24b855407abf607f233fbf3ee8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.928179 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a566438b-ca75-401d-a5ce-b9c9e15b9af7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9ff3c24b855407abf607f233fbf3ee8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.929508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a566438b-ca75-401d-a5ce-b9c9e15b9af7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9ff3c24b855407abf607f233fbf3ee8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.938700 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a566438b-ca75-401d-a5ce-b9c9e15b9af7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9ff3c24b855407abf607f233fbf3ee8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:37.948750 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 923/1845] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:37 2022] DELETE /identity/v3/users/d9ff3c24b855407abf607f233fbf3ee8 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:37.970362 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5824762c-f815-4a3a-b023-8a10406f6f1f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:37.984701 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:38.069952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907b3d2-02dd-49f2-968d-8e1007351476 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:38.070939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907b3d2-02dd-49f2-968d-8e1007351476 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:38.078040 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907b3d2-02dd-49f2-968d-8e1007351476 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:38.078470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6907b3d2-02dd-49f2-968d-8e1007351476 admin admin] PATH_INFO: `/v3/projects/600f27b5d2f94f839ee3cbfd06b69e74` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:38.080091 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907b3d2-02dd-49f2-968d-8e1007351476 admin admin] RBAC: Authorizing `identity:delete_project(project_id=600f27b5d2f94f839ee3cbfd06b69e74)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:38.083040 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6907b3d2-02dd-49f2-968d-8e1007351476 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:38.150731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6907b3d2-02dd-49f2-968d-8e1007351476 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '600f27b5d2f94f839ee3cbfd06b69e74', 'request_id': 'req-6907b3d2-02dd-49f2-968d-8e1007351476'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:38.165642 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6907b3d2-02dd-49f2-968d-8e1007351476 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '600f27b5d2f94f839ee3cbfd06b69e74', 'request_id': 'req-6907b3d2-02dd-49f2-968d-8e1007351476'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:38.176571 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6907b3d2-02dd-49f2-968d-8e1007351476 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '600f27b5d2f94f839ee3cbfd06b69e74', 'request_id': 'req-6907b3d2-02dd-49f2-968d-8e1007351476'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:38.193133 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6907b3d2-02dd-49f2-968d-8e1007351476 admin admin] The token cache is being invalidate because project 600f27b5d2f94f839ee3cbfd06b69e74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:38.193684 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6907b3d2-02dd-49f2-968d-8e1007351476 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:38.194602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6907b3d2-02dd-49f2-968d-8e1007351476 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:38.203261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 923/1846] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:37 2022] DELETE /identity/v3/projects/600f27b5d2f94f839ee3cbfd06b69e74 => generated 0 bytes in 336 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:38.205201 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6ccbf-9c1d-4a45-ab31-669c8f04854b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:38.205201 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6ccbf-9c1d-4a45-ab31-669c8f04854b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:38.205569 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26e6ccbf-9c1d-4a45-ab31-669c8f04854b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:38.227417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-26e6ccbf-9c1d-4a45-ab31-669c8f04854b None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:38.228997 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-26e6ccbf-9c1d-4a45-ab31-669c8f04854b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:38.313842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907b3d2-02dd-49f2-968d-8e1007351476 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:38.315450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6907b3d2-02dd-49f2-968d-8e1007351476 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:38.321756 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 924/1847] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:38.329355 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fa4d3-9458-4fc7-bf9d-9e844bb0ac53 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:38.329652 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fa4d3-9458-4fc7-bf9d-9e844bb0ac53 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:38.329936 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3fa4d3-9458-4fc7-bf9d-9e844bb0ac53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:38.339138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6907b3d2-02dd-49f2-968d-8e1007351476 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:38.356437 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ae3fa4d3-9458-4fc7-bf9d-9e844bb0ac53 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:38.357407 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ae3fa4d3-9458-4fc7-bf9d-9e844bb0ac53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:38.361731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8c83e3-2b95-4832-8235-11ce6ed95779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:38.362838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c83e3-2b95-4832-8235-11ce6ed95779 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:38.363249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c83e3-2b95-4832-8235-11ce6ed95779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:38.363617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8c83e3-2b95-4832-8235-11ce6ed95779 admin admin] PATH_INFO: `/v3/projects/0e826af69dec41a19d213faa3b49ec22` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:38.367759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c83e3-2b95-4832-8235-11ce6ed95779 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0e826af69dec41a19d213faa3b49ec22)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:38.370643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8c83e3-2b95-4832-8235-11ce6ed95779 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:38.372321 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 925/1848] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:38.376734 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fa4d3-9458-4fc7-bf9d-9e844bb0ac53 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:38.387753 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3fa4d3-9458-4fc7-bf9d-9e844bb0ac53 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:38.394609 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a639899-d559-4b69-9db4-b78201b1dd37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:38.395048 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a639899-d559-4b69-9db4-b78201b1dd37 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:38.395335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a639899-d559-4b69-9db4-b78201b1dd37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:38.395616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a639899-d559-4b69-9db4-b78201b1dd37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:38.396507 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a639899-d559-4b69-9db4-b78201b1dd37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:07:38.396827 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a639899-d559-4b69-9db4-b78201b1dd37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:38.398704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a639899-d559-4b69-9db4-b78201b1dd37 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:38.416872 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 926/1849] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:07:38.417994 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a639899-d559-4b69-9db4-b78201b1dd37 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:38.430763 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a639899-d559-4b69-9db4-b78201b1dd37 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:38.436570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc916add-bba2-4133-8e05-b0a05ecabdec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:38.437383 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc916add-bba2-4133-8e05-b0a05ecabdec admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:38.437745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc916add-bba2-4133-8e05-b0a05ecabdec admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:38.438099 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc916add-bba2-4133-8e05-b0a05ecabdec admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:38.439079 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc916add-bba2-4133-8e05-b0a05ecabdec admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:38.441278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc916add-bba2-4133-8e05-b0a05ecabdec admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:38.487178 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 927/1850] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:38 2022] POST /identity/v3/projects => generated 383 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:38.487307 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2e8c83e3-2b95-4832-8235-11ce6ed95779 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0e826af69dec41a19d213faa3b49ec22', 'request_id': 'req-2e8c83e3-2b95-4832-8235-11ce6ed95779'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:38.488323 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc916add-bba2-4133-8e05-b0a05ecabdec admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:38.494261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2e8c83e3-2b95-4832-8235-11ce6ed95779 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0e826af69dec41a19d213faa3b49ec22', 'request_id': 'req-2e8c83e3-2b95-4832-8235-11ce6ed95779'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:38.505137 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2e8c83e3-2b95-4832-8235-11ce6ed95779 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0e826af69dec41a19d213faa3b49ec22', 'request_id': 'req-2e8c83e3-2b95-4832-8235-11ce6ed95779'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:38.522945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2e8c83e3-2b95-4832-8235-11ce6ed95779 admin admin] The token cache is being invalidate because project 0e826af69dec41a19d213faa3b49ec22 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:38.523602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2e8c83e3-2b95-4832-8235-11ce6ed95779 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:38.525180 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2e8c83e3-2b95-4832-8235-11ce6ed95779 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:38.527811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 924/1851] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:38 2022] DELETE /identity/v3/projects/0e826af69dec41a19d213faa3b49ec22 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:38.533403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87a006-cd44-4806-a8ed-29cdd3d84b12 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:38.533918 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87a006-cd44-4806-a8ed-29cdd3d84b12 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:38.534409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e87a006-cd44-4806-a8ed-29cdd3d84b12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:38.566837 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7e87a006-cd44-4806-a8ed-29cdd3d84b12 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:38.568557 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7e87a006-cd44-4806-a8ed-29cdd3d84b12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:38.604925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:38.627694 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc916add-bba2-4133-8e05-b0a05ecabdec admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:38.646096 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5df49a-450e-4da7-be6f-bd189b4ff972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:38.647191 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5df49a-450e-4da7-be6f-bd189b4ff972 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:38.647864 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5df49a-450e-4da7-be6f-bd189b4ff972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:38.648365 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f5df49a-450e-4da7-be6f-bd189b4ff972 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:38.649417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5df49a-450e-4da7-be6f-bd189b4ff972 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:38.652743 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f5df49a-450e-4da7-be6f-bd189b4ff972 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:38.665778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=90,keystone:DELETE=5,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:38.666135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 925/1852] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:38.674095 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a61d6-9ff4-4ba1-8790-967c41d37af9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:38.674369 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a61d6-9ff4-4ba1-8790-967c41d37af9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:38.674769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-552a61d6-9ff4-4ba1-8790-967c41d37af9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:38.700111 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 928/1853] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:38 2022] POST /identity/v3/users => generated 292 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:07:38.704998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5df49a-450e-4da7-be6f-bd189b4ff972 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:38.710110 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9f5df49a-450e-4da7-be6f-bd189b4ff972 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:38.711420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-552a61d6-9ff4-4ba1-8790-967c41d37af9 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:38.712607 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-552a61d6-9ff4-4ba1-8790-967c41d37af9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:38.741378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f5df49a-450e-4da7-be6f-bd189b4ff972 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:38.746556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 926/1854] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:38.750802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafe833-02ad-4b31-bcc1-ebe7701dc88a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:38.750949 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a61d6-9ff4-4ba1-8790-967c41d37af9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:38.751650 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafe833-02ad-4b31-bcc1-ebe7701dc88a admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:38.752581 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafe833-02ad-4b31-bcc1-ebe7701dc88a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:38.752862 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cafe833-02ad-4b31-bcc1-ebe7701dc88a admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:38.753791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafe833-02ad-4b31-bcc1-ebe7701dc88a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:38.755779 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cafe833-02ad-4b31-bcc1-ebe7701dc88a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:38.772357 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-552a61d6-9ff4-4ba1-8790-967c41d37af9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:38.780844 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c14c8-65d2-4643-8814-ffe96e811d9e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:38.781615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c14c8-65d2-4643-8814-ffe96e811d9e admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:38.782046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c14c8-65d2-4643-8814-ffe96e811d9e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:38.782384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-895c14c8-65d2-4643-8814-ffe96e811d9e admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:38.783132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c14c8-65d2-4643-8814-ffe96e811d9e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:07:38.783446 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c14c8-65d2-4643-8814-ffe96e811d9e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:38.785057 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-895c14c8-65d2-4643-8814-ffe96e811d9e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:38.786393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 929/1855] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:38.789025 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafe833-02ad-4b31-bcc1-ebe7701dc88a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:38.806335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cafe833-02ad-4b31-bcc1-ebe7701dc88a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:38.810717 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 927/1856] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:07:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:07:38.811745 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c14c8-65d2-4643-8814-ffe96e811d9e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:38.815169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a29a7-bc78-4c74-9756-3681c56491f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:38.823171 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-895c14c8-65d2-4643-8814-ffe96e811d9e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:38.827582 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a29a7-bc78-4c74-9756-3681c56491f9 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:38.828448 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a29a7-bc78-4c74-9756-3681c56491f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:38.828876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741a29a7-bc78-4c74-9756-3681c56491f9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:38.829008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d9836-e272-455c-a3e5-143e3f0f4086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:38.829626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d9836-e272-455c-a3e5-143e3f0f4086 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:38.829918 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d9836-e272-455c-a3e5-143e3f0f4086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:38.830185 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-018d9836-e272-455c-a3e5-143e3f0f4086 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:38.830923 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d9836-e272-455c-a3e5-143e3f0f4086 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:38.832040 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a29a7-bc78-4c74-9756-3681c56491f9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:38.832816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-018d9836-e272-455c-a3e5-143e3f0f4086 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:38.834293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-741a29a7-bc78-4c74-9756-3681c56491f9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:38.848645 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 930/1857] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:38 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:38.852749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a29a7-bc78-4c74-9756-3681c56491f9 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:38.867419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-741a29a7-bc78-4c74-9756-3681c56491f9 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:38.885500 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197ec62-66fb-40f2-83a2-826d1c2798aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:38.886568 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197ec62-66fb-40f2-83a2-826d1c2798aa admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:38.886941 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197ec62-66fb-40f2-83a2-826d1c2798aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:38.887470 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e197ec62-66fb-40f2-83a2-826d1c2798aa admin admin] PATH_INFO: `/v3/projects/da60a417da4a4cbb94189abdbfa39d30/users/2d87c2248b96418896305deb4e0d6eff/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:38.906492 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 928/1858] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:38 2022] POST /identity/v3/projects => generated 385 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:38.908437 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d9836-e272-455c-a3e5-143e3f0f4086 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:38.910525 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-018d9836-e272-455c-a3e5-143e3f0f4086 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:38.912813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197ec62-66fb-40f2-83a2-826d1c2798aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=da60a417da4a4cbb94189abdbfa39d30, user_id=2d87c2248b96418896305deb4e0d6eff, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:38.915523 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e197ec62-66fb-40f2-83a2-826d1c2798aa admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:38.931646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 931/1859] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:38 2022] PUT /identity/v3/projects/da60a417da4a4cbb94189abdbfa39d30/users/2d87c2248b96418896305deb4e0d6eff/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:38.932471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197ec62-66fb-40f2-83a2-826d1c2798aa admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:38.983335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-018d9836-e272-455c-a3e5-143e3f0f4086 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:39.006616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e197ec62-66fb-40f2-83a2-826d1c2798aa admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:39.011323 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff2ee36-5609-40a6-8c38-c685e7ada0c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:39.012000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2ee36-5609-40a6-8c38-c685e7ada0c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:39.012284 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2ee36-5609-40a6-8c38-c685e7ada0c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:39.012673 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff2ee36-5609-40a6-8c38-c685e7ada0c1 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:39.013256 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff2ee36-5609-40a6-8c38-c685e7ada0c1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:39.015263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff2ee36-5609-40a6-8c38-c685e7ada0c1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:39.039252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 932/1860] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:38 2022] POST /identity/v3/users => generated 293 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:39.039252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f885613-2881-4e95-969e-e29024892e14 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:39.039252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f885613-2881-4e95-969e-e29024892e14 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:39.039252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f885613-2881-4e95-969e-e29024892e14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:39.041407 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f1603-c4a3-4a4b-b000-0bb8ad971829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:39.042276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f1603-c4a3-4a4b-b000-0bb8ad971829 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:39.042659 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f1603-c4a3-4a4b-b000-0bb8ad971829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:39.042941 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c8f1603-c4a3-4a4b-b000-0bb8ad971829 admin admin] PATH_INFO: `/v3/projects/bd04a81f2c5d4906a17dfd2925de70e0` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:39.044203 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f1603-c4a3-4a4b-b000-0bb8ad971829 admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd04a81f2c5d4906a17dfd2925de70e0)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:39.046158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c8f1603-c4a3-4a4b-b000-0bb8ad971829 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:39.073320 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1f885613-2881-4e95-969e-e29024892e14 None None] MFA Rules not processed for user `2d87c2248b96418896305deb4e0d6eff`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:39.074749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1f885613-2881-4e95-969e-e29024892e14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:39.113143 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:39.132401 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6c8f1603-c4a3-4a4b-b000-0bb8ad971829 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd04a81f2c5d4906a17dfd2925de70e0', 'request_id': 'req-6c8f1603-c4a3-4a4b-b000-0bb8ad971829'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:39.139801 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6c8f1603-c4a3-4a4b-b000-0bb8ad971829 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd04a81f2c5d4906a17dfd2925de70e0', 'request_id': 'req-6c8f1603-c4a3-4a4b-b000-0bb8ad971829'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:39.153575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6c8f1603-c4a3-4a4b-b000-0bb8ad971829 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd04a81f2c5d4906a17dfd2925de70e0', 'request_id': 'req-6c8f1603-c4a3-4a4b-b000-0bb8ad971829'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:39.175085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6c8f1603-c4a3-4a4b-b000-0bb8ad971829 admin admin] The token cache is being invalidate because project bd04a81f2c5d4906a17dfd2925de70e0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:39.175627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6c8f1603-c4a3-4a4b-b000-0bb8ad971829 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:39.177266 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-6c8f1603-c4a3-4a4b-b000-0bb8ad971829 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:39.179070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 929/1861] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:38 2022] DELETE /identity/v3/projects/bd04a81f2c5d4906a17dfd2925de70e0 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:39.180098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f1603-c4a3-4a4b-b000-0bb8ad971829 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:39.184757 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6c8f1603-c4a3-4a4b-b000-0bb8ad971829 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:39.204091 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 933/1862] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:39 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:39.205360 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100d312-6794-431e-b8e5-77028b0d3570 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:39.205673 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100d312-6794-431e-b8e5-77028b0d3570 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:39.205941 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b100d312-6794-431e-b8e5-77028b0d3570 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:39.237118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b100d312-6794-431e-b8e5-77028b0d3570 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:39.238518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b100d312-6794-431e-b8e5-77028b0d3570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:39.274656 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:39.284079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c8f1603-c4a3-4a4b-b000-0bb8ad971829 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:39.289617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4569940-1df3-4ee4-a046-2209224c8635 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:39.290346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4569940-1df3-4ee4-a046-2209224c8635 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:39.290637 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4569940-1df3-4ee4-a046-2209224c8635 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:39.290916 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4569940-1df3-4ee4-a046-2209224c8635 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:39.291651 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4569940-1df3-4ee4-a046-2209224c8635 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:39.293306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4569940-1df3-4ee4-a046-2209224c8635 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:39.304335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 930/1863] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:39.305123 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4569940-1df3-4ee4-a046-2209224c8635 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:39.306837 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f4569940-1df3-4ee4-a046-2209224c8635 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:39.314768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 934/1864] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:39.315759 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100d312-6794-431e-b8e5-77028b0d3570 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:39.320021 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4569940-1df3-4ee4-a046-2209224c8635 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:39.324695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ec3e2-64f3-4fac-ae10-374f103cf14f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:39.325353 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ec3e2-64f3-4fac-ae10-374f103cf14f admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:39.325527 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b100d312-6794-431e-b8e5-77028b0d3570 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:39.325626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ec3e2-64f3-4fac-ae10-374f103cf14f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:39.325901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4ec3e2-64f3-4fac-ae10-374f103cf14f admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:39.326629 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ec3e2-64f3-4fac-ae10-374f103cf14f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:39.328571 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4ec3e2-64f3-4fac-ae10-374f103cf14f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:39.330678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-108f607e-5eb4-4ecf-aef5-96e3d86acd81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:39.331455 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f607e-5eb4-4ecf-aef5-96e3d86acd81 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:39.331879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f607e-5eb4-4ecf-aef5-96e3d86acd81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:39.332263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-108f607e-5eb4-4ecf-aef5-96e3d86acd81 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:39.333105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108f607e-5eb4-4ecf-aef5-96e3d86acd81 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:39.335029 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-108f607e-5eb4-4ecf-aef5-96e3d86acd81 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:39.345958 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 935/1865] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:39.349512 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf47f0-0dc1-42db-86e0-2f48da51e7d0 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:39.350193 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf47f0-0dc1-42db-86e0-2f48da51e7d0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:39.350193 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99cf47f0-0dc1-42db-86e0-2f48da51e7d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:39.362490 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 931/1866] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:39 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:39.362490 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ec3e2-64f3-4fac-ae10-374f103cf14f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:39.370515 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-99cf47f0-0dc1-42db-86e0-2f48da51e7d0 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:39.371736 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-99cf47f0-0dc1-42db-86e0-2f48da51e7d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:39.435700 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4ec3e2-64f3-4fac-ae10-374f103cf14f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:39.443855 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99694bbf-c881-4840-9b74-e655d1f75b6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:39.444765 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99694bbf-c881-4840-9b74-e655d1f75b6c admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:39.445054 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99694bbf-c881-4840-9b74-e655d1f75b6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:39.445335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99694bbf-c881-4840-9b74-e655d1f75b6c admin admin] PATH_INFO: `/v3/projects/5b52e9a898274997b56e86800aa4f04f/users/ce76587042e64fc2af6862d2af6d86f1/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:39.447080 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 936/1867] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:39.448273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf47f0-0dc1-42db-86e0-2f48da51e7d0 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:39.460349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99cf47f0-0dc1-42db-86e0-2f48da51e7d0 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:39.464590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99694bbf-c881-4840-9b74-e655d1f75b6c admin admin] RBAC: Authorizing `identity:create_grant(project_id=5b52e9a898274997b56e86800aa4f04f, user_id=ce76587042e64fc2af6862d2af6d86f1, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:39.465393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-799d5a31-ad62-4252-ad3e-250ea17f524b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:39.466182 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d5a31-ad62-4252-ad3e-250ea17f524b admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:39.466390 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d5a31-ad62-4252-ad3e-250ea17f524b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:39.466673 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-799d5a31-ad62-4252-ad3e-250ea17f524b admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:39.467637 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d5a31-ad62-4252-ad3e-250ea17f524b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:39.468065 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99694bbf-c881-4840-9b74-e655d1f75b6c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:39.471430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-799d5a31-ad62-4252-ad3e-250ea17f524b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:39.488719 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 932/1868] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:39 2022] PUT /identity/v3/projects/5b52e9a898274997b56e86800aa4f04f/users/ce76587042e64fc2af6862d2af6d86f1/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:39.490078 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99694bbf-c881-4840-9b74-e655d1f75b6c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:39.496986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 937/1869] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:39 2022] POST /identity/v3/users => generated 296 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:39.502065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2415117-e7ec-4123-87b4-ccfcaddbaeb9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:39.502948 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2415117-e7ec-4123-87b4-ccfcaddbaeb9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:39.504715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2415117-e7ec-4123-87b4-ccfcaddbaeb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:39.524899 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=5,keystone:SELECT=92,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:39.544369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e2415117-e7ec-4123-87b4-ccfcaddbaeb9 None None] MFA Rules not processed for user `ce76587042e64fc2af6862d2af6d86f1`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:39.545611 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e2415117-e7ec-4123-87b4-ccfcaddbaeb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:39.575568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99694bbf-c881-4840-9b74-e655d1f75b6c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:39.581197 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523177f-fb12-46fc-9efe-9baa9ec36452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:39.582035 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523177f-fb12-46fc-9efe-9baa9ec36452 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:39.582462 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523177f-fb12-46fc-9efe-9baa9ec36452 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:39.582843 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5523177f-fb12-46fc-9efe-9baa9ec36452 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:39.583716 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523177f-fb12-46fc-9efe-9baa9ec36452 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:07:39.584134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523177f-fb12-46fc-9efe-9baa9ec36452 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:39.586133 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5523177f-fb12-46fc-9efe-9baa9ec36452 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:39.598992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 933/1870] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:07:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:07:39.599829 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523177f-fb12-46fc-9efe-9baa9ec36452 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:39.611326 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5523177f-fb12-46fc-9efe-9baa9ec36452 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:39.616121 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32e972-2fd2-4595-8a7b-9b2e21329f62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:39.616799 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32e972-2fd2-4595-8a7b-9b2e21329f62 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:39.617089 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32e972-2fd2-4595-8a7b-9b2e21329f62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:39.617370 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce32e972-2fd2-4595-8a7b-9b2e21329f62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:39.618274 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32e972-2fd2-4595-8a7b-9b2e21329f62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:39.619924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce32e972-2fd2-4595-8a7b-9b2e21329f62 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:39.628195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 938/1871] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:39.629052 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2415117-e7ec-4123-87b4-ccfcaddbaeb9 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:39.644883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2415117-e7ec-4123-87b4-ccfcaddbaeb9 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:39.645071 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 934/1872] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:39.645071 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32e972-2fd2-4595-8a7b-9b2e21329f62 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:39.645071 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ce32e972-2fd2-4595-8a7b-9b2e21329f62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:39.650434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91b200-f24a-4a22-abc4-893be8a97f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:39.651136 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91b200-f24a-4a22-abc4-893be8a97f17 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:39.651423 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91b200-f24a-4a22-abc4-893be8a97f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:39.652016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc91b200-f24a-4a22-abc4-893be8a97f17 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:39.652402 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91b200-f24a-4a22-abc4-893be8a97f17 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:39.654346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc91b200-f24a-4a22-abc4-893be8a97f17 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:39.685247 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 939/1873] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:39 2022] POST /identity/v3/projects => generated 381 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:39.686133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91b200-f24a-4a22-abc4-893be8a97f17 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:39.697990 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:39.707373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce32e972-2fd2-4595-8a7b-9b2e21329f62 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:39.755473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc91b200-f24a-4a22-abc4-893be8a97f17 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:39.760930 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b145205-0a83-43cc-bdba-e89ec096e36e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:39.761588 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b145205-0a83-43cc-bdba-e89ec096e36e service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:39.761737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c196245-aed7-4851-b915-8d2f21453d85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:39.761871 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b145205-0a83-43cc-bdba-e89ec096e36e service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:39.762132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b145205-0a83-43cc-bdba-e89ec096e36e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:39.762868 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196245-aed7-4851-b915-8d2f21453d85 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:39.763169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196245-aed7-4851-b915-8d2f21453d85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:39.763450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c196245-aed7-4851-b915-8d2f21453d85 admin admin] PATH_INFO: `/v3/projects/befd285c3f78447b931814fbb57ee254/users/f4f120d756614e1096353deefc9d5b7a/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:39.783400 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c196245-aed7-4851-b915-8d2f21453d85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=befd285c3f78447b931814fbb57ee254, user_id=f4f120d756614e1096353deefc9d5b7a, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:39.786257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c196245-aed7-4851-b915-8d2f21453d85 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:39.805278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 940/1874] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:39 2022] PUT /identity/v3/projects/befd285c3f78447b931814fbb57ee254/users/f4f120d756614e1096353deefc9d5b7a/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:39.806020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c196245-aed7-4851-b915-8d2f21453d85 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:39.835256 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b145205-0a83-43cc-bdba-e89ec096e36e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:39.839129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b145205-0a83-43cc-bdba-e89ec096e36e service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:39.851437 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 935/1875] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:39 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 209 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:39.852086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b145205-0a83-43cc-bdba-e89ec096e36e service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:39.864187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b145205-0a83-43cc-bdba-e89ec096e36e service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:39.874456 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:39.931593 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52003b69-757f-4e5a-8c56-ae8a54094aa2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:39.932285 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52003b69-757f-4e5a-8c56-ae8a54094aa2 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:39.932575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52003b69-757f-4e5a-8c56-ae8a54094aa2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:39.932919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52003b69-757f-4e5a-8c56-ae8a54094aa2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:39.933968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52003b69-757f-4e5a-8c56-ae8a54094aa2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:39.934339 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c196245-aed7-4851-b915-8d2f21453d85 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:39.936200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52003b69-757f-4e5a-8c56-ae8a54094aa2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:39.941666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbac3f0-72af-4978-a93a-cbbae06de42e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:39.943381 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbac3f0-72af-4978-a93a-cbbae06de42e admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:39.943673 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbac3f0-72af-4978-a93a-cbbae06de42e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:39.943954 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bbac3f0-72af-4978-a93a-cbbae06de42e admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:39.944662 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbac3f0-72af-4978-a93a-cbbae06de42e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:39.947940 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bbac3f0-72af-4978-a93a-cbbae06de42e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:39.948593 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 936/1876] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:39.952312 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52003b69-757f-4e5a-8c56-ae8a54094aa2 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:39.965483 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52003b69-757f-4e5a-8c56-ae8a54094aa2 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:39.972566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312ff74b-a5b5-413f-b141-b7cf1ece5480 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:39.973570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ff74b-a5b5-413f-b141-b7cf1ece5480 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:39.973938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ff74b-a5b5-413f-b141-b7cf1ece5480 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:39.974451 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312ff74b-a5b5-413f-b141-b7cf1ece5480 admin admin] PATH_INFO: `/v3/projects/befd285c3f78447b931814fbb57ee254/users/f4f120d756614e1096353deefc9d5b7a/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:39.976736 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ff74b-a5b5-413f-b141-b7cf1ece5480 admin admin] RBAC: Authorizing `identity:create_grant(project_id=befd285c3f78447b931814fbb57ee254, user_id=f4f120d756614e1096353deefc9d5b7a, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:39.979697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312ff74b-a5b5-413f-b141-b7cf1ece5480 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:39.988423 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 941/1877] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:39 2022] POST /identity/v3/users => generated 291 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:39.989540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbac3f0-72af-4978-a93a-cbbae06de42e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:39.993613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 937/1878] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:39 2022] PUT /identity/v3/projects/befd285c3f78447b931814fbb57ee254/users/f4f120d756614e1096353deefc9d5b7a/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:40.004485 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6518789-e462-496b-a6b7-58a1d7863807 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:40.004786 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6518789-e462-496b-a6b7-58a1d7863807 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:40.005070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6518789-e462-496b-a6b7-58a1d7863807 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:40.021082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bbac3f0-72af-4978-a93a-cbbae06de42e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:40.051177 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e6518789-e462-496b-a6b7-58a1d7863807 None None] MFA Rules not processed for user `f4f120d756614e1096353deefc9d5b7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:40.052712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e6518789-e462-496b-a6b7-58a1d7863807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:40.135784 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4e082-5aae-4642-8a6c-c2c96001ffcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:40.136782 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4e082-5aae-4642-8a6c-c2c96001ffcb admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:40.137078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4e082-5aae-4642-8a6c-c2c96001ffcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:40.137474 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db4e082-5aae-4642-8a6c-c2c96001ffcb admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:40.138223 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4e082-5aae-4642-8a6c-c2c96001ffcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:40.141416 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db4e082-5aae-4642-8a6c-c2c96001ffcb admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:40.148801 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 938/1879] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:40.162374 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 942/1880] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:39 2022] GET /identity/v3/roles => generated 1420 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:40.162735 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6518789-e462-496b-a6b7-58a1d7863807 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:40.162902 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e6518789-e462-496b-a6b7-58a1d7863807 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:40.163886 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4e082-5aae-4642-8a6c-c2c96001ffcb admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:40.175322 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db4e082-5aae-4642-8a6c-c2c96001ffcb admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:40.181517 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fccc6-fc2f-4451-965a-fcd045db8fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:40.182609 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fccc6-fc2f-4451-965a-fcd045db8fae admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:40.183055 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fccc6-fc2f-4451-965a-fcd045db8fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:40.183914 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d74fccc6-fc2f-4451-965a-fcd045db8fae admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:40.185185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fccc6-fc2f-4451-965a-fcd045db8fae admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:40.187998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d74fccc6-fc2f-4451-965a-fcd045db8fae admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:40.210913 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 943/1881] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:40.215127 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fccc6-fc2f-4451-965a-fcd045db8fae admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:40.231265 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d74fccc6-fc2f-4451-965a-fcd045db8fae admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:40.236670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073e95d-0bb0-4487-b5c5-bee19007c29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:40.237608 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073e95d-0bb0-4487-b5c5-bee19007c29c admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:40.237914 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073e95d-0bb0-4487-b5c5-bee19007c29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:40.238183 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6073e95d-0bb0-4487-b5c5-bee19007c29c admin admin] PATH_INFO: `/v3/projects/96d67ee8493e495baeec2d70bf79224a/users/754c411f631449c09819c9e4fc06b0cb/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:40.241050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:40.257893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6518789-e462-496b-a6b7-58a1d7863807 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:40.263682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65132af5-3f64-483b-97b0-6145a024cc3b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:40.264328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6073e95d-0bb0-4487-b5c5-bee19007c29c admin admin] RBAC: Authorizing `identity:create_grant(project_id=96d67ee8493e495baeec2d70bf79224a, user_id=754c411f631449c09819c9e4fc06b0cb, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:40.264621 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65132af5-3f64-483b-97b0-6145a024cc3b service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:40.265006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65132af5-3f64-483b-97b0-6145a024cc3b service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:40.265953 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65132af5-3f64-483b-97b0-6145a024cc3b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:40.268910 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6073e95d-0bb0-4487-b5c5-bee19007c29c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:40.282095 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65132af5-3f64-483b-97b0-6145a024cc3b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:40.284900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65132af5-3f64-483b-97b0-6145a024cc3b service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:40.297074 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 939/1883] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:40 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:07:40.302690 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 944/1883] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:40 2022] PUT /identity/v3/projects/96d67ee8493e495baeec2d70bf79224a/users/754c411f631449c09819c9e4fc06b0cb/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 79 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:40.313322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a9390-4b1f-49d4-9bdb-fddccab8e879 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:40.314392 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a9390-4b1f-49d4-9bdb-fddccab8e879 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:40.315213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-252a9390-4b1f-49d4-9bdb-fddccab8e879 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:40.352995 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-252a9390-4b1f-49d4-9bdb-fddccab8e879 None None] MFA Rules not processed for user `754c411f631449c09819c9e4fc06b0cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:40.354520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-252a9390-4b1f-49d4-9bdb-fddccab8e879 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:40.369822 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:40.434567 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073e95d-0bb0-4487-b5c5-bee19007c29c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:40.448349 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 940/1884] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:40 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:40.452691 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a9390-4b1f-49d4-9bdb-fddccab8e879 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:40.506373 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6073e95d-0bb0-4487-b5c5-bee19007c29c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:40.510444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7e889-0a22-45a8-89cf-5e1915f26e45 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:40.511204 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7e889-0a22-45a8-89cf-5e1915f26e45 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:40.511498 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7e889-0a22-45a8-89cf-5e1915f26e45 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:40.511770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba7e889-0a22-45a8-89cf-5e1915f26e45 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:40.513670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4ba7e889-0a22-45a8-89cf-5e1915f26e45 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:40.531526 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-252a9390-4b1f-49d4-9bdb-fddccab8e879 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:40.536851 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4239a20-4b9f-473e-bacd-aebeb1125e36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:40.537703 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4239a20-4b9f-473e-bacd-aebeb1125e36 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:40.538029 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4239a20-4b9f-473e-bacd-aebeb1125e36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:40.538251 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4239a20-4b9f-473e-bacd-aebeb1125e36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:40.538985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4239a20-4b9f-473e-bacd-aebeb1125e36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:40.541211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4239a20-4b9f-473e-bacd-aebeb1125e36 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:40.584354 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 941/1885] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:40 2022] POST /identity/v3/projects => generated 379 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:07:40.586947 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7e889-0a22-45a8-89cf-5e1915f26e45 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:40.588043 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba7e889-0a22-45a8-89cf-5e1915f26e45 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:40.589004 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4239a20-4b9f-473e-bacd-aebeb1125e36 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:40.599873 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 945/1886] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:40 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:07:40.738555 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4239a20-4b9f-473e-bacd-aebeb1125e36 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:40.744379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49b537-e6cc-449d-9059-425defe5f34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:40.745405 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49b537-e6cc-449d-9059-425defe5f34b admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:40.746008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49b537-e6cc-449d-9059-425defe5f34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:40.746540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c49b537-e6cc-449d-9059-425defe5f34b admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:40.747496 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c49b537-e6cc-449d-9059-425defe5f34b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:40.750957 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c49b537-e6cc-449d-9059-425defe5f34b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:40.776806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 942/1887] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:40 2022] POST /identity/v3/users => generated 296 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:40.787645 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7e889-0a22-45a8-89cf-5e1915f26e45 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:40.800928 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba7e889-0a22-45a8-89cf-5e1915f26e45 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:40.806781 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d129c4-5f88-47d2-9da7-8d81cf81fbdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:40.807471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d129c4-5f88-47d2-9da7-8d81cf81fbdd admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:40.807836 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d129c4-5f88-47d2-9da7-8d81cf81fbdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:40.808225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d129c4-5f88-47d2-9da7-8d81cf81fbdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:40.809674 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d129c4-5f88-47d2-9da7-8d81cf81fbdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:40.811105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6d129c4-5f88-47d2-9da7-8d81cf81fbdd admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:40.825628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 946/1888] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:40.830866 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49b537-e6cc-449d-9059-425defe5f34b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:40.847414 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c49b537-e6cc-449d-9059-425defe5f34b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:40.851722 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c0151-00aa-4184-bef0-956729e81f16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:40.852626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c0151-00aa-4184-bef0-956729e81f16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:40.852914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c0151-00aa-4184-bef0-956729e81f16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:40.853193 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c54c0151-00aa-4184-bef0-956729e81f16 admin admin] PATH_INFO: `/v3/projects/316691dd906f4272b392f532c00f4ac9/users/00f0195a2f6b47bdabd8c3b97838fa23/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:40.874125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54c0151-00aa-4184-bef0-956729e81f16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=316691dd906f4272b392f532c00f4ac9, user_id=00f0195a2f6b47bdabd8c3b97838fa23, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:40.876990 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c54c0151-00aa-4184-bef0-956729e81f16 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:40.892314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 943/1889] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:40 2022] PUT /identity/v3/projects/316691dd906f4272b392f532c00f4ac9/users/00f0195a2f6b47bdabd8c3b97838fa23/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:40.896665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d129c4-5f88-47d2-9da7-8d81cf81fbdd admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:40.908767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6d129c4-5f88-47d2-9da7-8d81cf81fbdd admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:40.952507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c0151-00aa-4184-bef0-956729e81f16 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:40.959732 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:40.979068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb66aa-18ae-42dc-9ff3-c25965827ba2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:40.979749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb66aa-18ae-42dc-9ff3-c25965827ba2 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:40.980034 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb66aa-18ae-42dc-9ff3-c25965827ba2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:40.980309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47cb66aa-18ae-42dc-9ff3-c25965827ba2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:40.981118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb66aa-18ae-42dc-9ff3-c25965827ba2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:40.982839 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47cb66aa-18ae-42dc-9ff3-c25965827ba2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:40.997581 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 947/1890] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:40 2022] GET /identity/v3/roles => generated 1420 bytes in 101 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:40.998485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb66aa-18ae-42dc-9ff3-c25965827ba2 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:41.009437 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47cb66aa-18ae-42dc-9ff3-c25965827ba2 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:41.014360 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8764d5c8-51ec-4911-87d7-ecc2db1787c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:41.017580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764d5c8-51ec-4911-87d7-ecc2db1787c6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:41.017580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764d5c8-51ec-4911-87d7-ecc2db1787c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:41.017580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8764d5c8-51ec-4911-87d7-ecc2db1787c6 admin admin] PATH_INFO: `/v3/projects/316691dd906f4272b392f532c00f4ac9/users/00f0195a2f6b47bdabd8c3b97838fa23/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:41.018475 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764d5c8-51ec-4911-87d7-ecc2db1787c6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=316691dd906f4272b392f532c00f4ac9, user_id=00f0195a2f6b47bdabd8c3b97838fa23, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:41.021160 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8764d5c8-51ec-4911-87d7-ecc2db1787c6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:41.027049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c54c0151-00aa-4184-bef0-956729e81f16 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:41.037858 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 948/1891] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:40 2022] PUT /identity/v3/projects/316691dd906f4272b392f532c00f4ac9/users/00f0195a2f6b47bdabd8c3b97838fa23/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:41.048860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fef988-fdbd-42d4-a21e-9f821ee5cbae None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:41.049204 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fef988-fdbd-42d4-a21e-9f821ee5cbae None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:41.049436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5fef988-fdbd-42d4-a21e-9f821ee5cbae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:41.070383 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:41.092991 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a5fef988-fdbd-42d4-a21e-9f821ee5cbae None None] MFA Rules not processed for user `00f0195a2f6b47bdabd8c3b97838fa23`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:41.094248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a5fef988-fdbd-42d4-a21e-9f821ee5cbae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:41.116585 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d97a8fa-4c60-4318-9f80-29a19f487068 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:41.117494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97a8fa-4c60-4318-9f80-29a19f487068 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:41.117784 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97a8fa-4c60-4318-9f80-29a19f487068 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:41.118063 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d97a8fa-4c60-4318-9f80-29a19f487068 admin admin] PATH_INFO: `/v3/users/2d87c2248b96418896305deb4e0d6eff` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:41.120919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97a8fa-4c60-4318-9f80-29a19f487068 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d87c2248b96418896305deb4e0d6eff)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:41.123082 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d97a8fa-4c60-4318-9f80-29a19f487068 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:41.213649 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9d97a8fa-4c60-4318-9f80-29a19f487068 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d87c2248b96418896305deb4e0d6eff'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:41.214656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9d97a8fa-4c60-4318-9f80-29a19f487068 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d87c2248b96418896305deb4e0d6eff'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:41.225650 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9d97a8fa-4c60-4318-9f80-29a19f487068 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d87c2248b96418896305deb4e0d6eff'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:41.227591 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9d97a8fa-4c60-4318-9f80-29a19f487068 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d87c2248b96418896305deb4e0d6eff'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:41.234969 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 949/1892] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:41 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:41.244289 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fef988-fdbd-42d4-a21e-9f821ee5cbae None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:41.259957 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9d97a8fa-4c60-4318-9f80-29a19f487068 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d87c2248b96418896305deb4e0d6eff'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:41.279021 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 944/1893] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:40 2022] DELETE /identity/v3/users/2d87c2248b96418896305deb4e0d6eff => generated 0 bytes in 327 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:41.285076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b8644-2dde-48c0-ba3f-048acf0aa557 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:41.285076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b8644-2dde-48c0-ba3f-048acf0aa557 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:41.285076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-102b8644-2dde-48c0-ba3f-048acf0aa557 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:41.303206 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-102b8644-2dde-48c0-ba3f-048acf0aa557 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:41.303888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-102b8644-2dde-48c0-ba3f-048acf0aa557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:41.310651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5fef988-fdbd-42d4-a21e-9f821ee5cbae None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:41.335725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:41.371575 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bc27f-c54a-48f0-abf9-b29c5bb35e9f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:41.372934 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bc27f-c54a-48f0-abf9-b29c5bb35e9f service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:41.374111 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bc27f-c54a-48f0-abf9-b29c5bb35e9f service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:41.375228 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f96bc27f-c54a-48f0-abf9-b29c5bb35e9f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:41.377244 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f96bc27f-c54a-48f0-abf9-b29c5bb35e9f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:41.380380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 945/1894] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:41.391548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b8644-2dde-48c0-ba3f-048acf0aa557 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:41.394021 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-102b8644-2dde-48c0-ba3f-048acf0aa557 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:41.473900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-102b8644-2dde-48c0-ba3f-048acf0aa557 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:41.479130 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02df04-2320-43a5-8352-96fc5b87b3f3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:41.480245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02df04-2320-43a5-8352-96fc5b87b3f3 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:41.481266 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02df04-2320-43a5-8352-96fc5b87b3f3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:41.481266 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e02df04-2320-43a5-8352-96fc5b87b3f3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:41.510116 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96bc27f-c54a-48f0-abf9-b29c5bb35e9f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:41.511794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f96bc27f-c54a-48f0-abf9-b29c5bb35e9f service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:41.515377 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e02df04-2320-43a5-8352-96fc5b87b3f3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:41.517663 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e02df04-2320-43a5-8352-96fc5b87b3f3 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:41.528548 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:41.539609 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 950/1895] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:41 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:07:41.540767 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 946/1896] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:07:41.990485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bc27f-c54a-48f0-abf9-b29c5bb35e9f service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:42.000596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f96bc27f-c54a-48f0-abf9-b29c5bb35e9f service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:42.006898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12acfa91-fb26-49ed-996f-28fd3f74622c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f0195a2f6b47bdabd8c3b97838fa23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316691dd906f4272b392f532c00f4ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:42.008281 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acfa91-fb26-49ed-996f-28fd3f74622c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:42.008577 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acfa91-fb26-49ed-996f-28fd3f74622c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:42.009244 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12acfa91-fb26-49ed-996f-28fd3f74622c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:42.009623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12acfa91-fb26-49ed-996f-28fd3f74622c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:07:42.009945 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12acfa91-fb26-49ed-996f-28fd3f74622c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:42.011635 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12acfa91-fb26-49ed-996f-28fd3f74622c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:42.028094 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 951/1897] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:07:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:07:42.029339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02df04-2320-43a5-8352-96fc5b87b3f3 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:42.044119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e02df04-2320-43a5-8352-96fc5b87b3f3 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:42.052761 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca763c-f700-460e-a2fe-a9db54fac64b tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f0195a2f6b47bdabd8c3b97838fa23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316691dd906f4272b392f532c00f4ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:42.053565 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca763c-f700-460e-a2fe-a9db54fac64b tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:42.053968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca763c-f700-460e-a2fe-a9db54fac64b tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:42.054867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca763c-f700-460e-a2fe-a9db54fac64b tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:42.055851 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca763c-f700-460e-a2fe-a9db54fac64b tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:42.058935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca763c-f700-460e-a2fe-a9db54fac64b tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:42.091470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 947/1898] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:42 2022] POST /identity/v3/projects => generated 353 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:42.095727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12acfa91-fb26-49ed-996f-28fd3f74622c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:42.107011 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12acfa91-fb26-49ed-996f-28fd3f74622c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:42.189820 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a857999-7f7b-4b0f-aaae-1c6407d2e128 tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f0195a2f6b47bdabd8c3b97838fa23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316691dd906f4272b392f532c00f4ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:42.191159 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a857999-7f7b-4b0f-aaae-1c6407d2e128 tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:42.191522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a857999-7f7b-4b0f-aaae-1c6407d2e128 tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:42.191901 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a857999-7f7b-4b0f-aaae-1c6407d2e128 tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:42.193046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a857999-7f7b-4b0f-aaae-1c6407d2e128 tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:07:42.193887 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a857999-7f7b-4b0f-aaae-1c6407d2e128 tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:42.195459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a857999-7f7b-4b0f-aaae-1c6407d2e128 tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:42.208838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 952/1899] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:07:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:07:42.250284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca763c-f700-460e-a2fe-a9db54fac64b tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:42.269044 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca763c-f700-460e-a2fe-a9db54fac64b tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:42.273821 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ba599-fe89-4d4d-9bb0-303b4cca284c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '00f0195a2f6b47bdabd8c3b97838fa23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '316691dd906f4272b392f532c00f4ac9', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:42.274732 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ba599-fe89-4d4d-9bb0-303b4cca284c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:42.275025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ba599-fe89-4d4d-9bb0-303b4cca284c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:42.275334 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0ba599-fe89-4d4d-9bb0-303b4cca284c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] PATH_INFO: `/v3/projects/01b2bf1e8f654e1690be4a0314dbd65f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:42.276533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ba599-fe89-4d4d-9bb0-303b4cca284c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] RBAC: Authorizing `identity:delete_project(project_id=01b2bf1e8f654e1690be4a0314dbd65f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:42.278521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0ba599-fe89-4d4d-9bb0-303b4cca284c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:42.327015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9a0ba599-fe89-4d4d-9bb0-303b4cca284c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01b2bf1e8f654e1690be4a0314dbd65f', 'request_id': 'req-9a0ba599-fe89-4d4d-9bb0-303b4cca284c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:42.333578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9a0ba599-fe89-4d4d-9bb0-303b4cca284c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01b2bf1e8f654e1690be4a0314dbd65f', 'request_id': 'req-9a0ba599-fe89-4d4d-9bb0-303b4cca284c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:42.341115 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9a0ba599-fe89-4d4d-9bb0-303b4cca284c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01b2bf1e8f654e1690be4a0314dbd65f', 'request_id': 'req-9a0ba599-fe89-4d4d-9bb0-303b4cca284c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:42.354155 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9a0ba599-fe89-4d4d-9bb0-303b4cca284c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] The token cache is being invalidate because project 01b2bf1e8f654e1690be4a0314dbd65f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:42.354687 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9a0ba599-fe89-4d4d-9bb0-303b4cca284c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:42.355307 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9a0ba599-fe89-4d4d-9bb0-303b4cca284c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:42.356918 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 948/1900] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:42 2022] DELETE /identity/v3/projects/01b2bf1e8f654e1690be4a0314dbd65f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:42.533060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a857999-7f7b-4b0f-aaae-1c6407d2e128 tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:42.535289 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9a857999-7f7b-4b0f-aaae-1c6407d2e128 tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:42.603420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ba599-fe89-4d4d-9bb0-303b4cca284c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:42.605585 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9a0ba599-fe89-4d4d-9bb0-303b4cca284c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:42.633699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a857999-7f7b-4b0f-aaae-1c6407d2e128 tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:42.662819 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:42.713577 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb56918-4417-4233-9deb-e93379cd00d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:42.714889 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb56918-4417-4233-9deb-e93379cd00d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:42.715267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb56918-4417-4233-9deb-e93379cd00d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:42.715620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb56918-4417-4233-9deb-e93379cd00d7 admin admin] PATH_INFO: `/v3/projects/da60a417da4a4cbb94189abdbfa39d30` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:42.717181 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb56918-4417-4233-9deb-e93379cd00d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da60a417da4a4cbb94189abdbfa39d30)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:42.720312 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb56918-4417-4233-9deb-e93379cd00d7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:42.725627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0ba599-fe89-4d4d-9bb0-303b4cca284c tempest-VolumeQuotaClassesTest-229727147 tempest-VolumeQuotaClassesTest-229727147-project-admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:42.733678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd68e4a-31f5-4d9c-bfb8-822fa92cd84d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:42.734895 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd68e4a-31f5-4d9c-bfb8-822fa92cd84d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:42.735359 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd68e4a-31f5-4d9c-bfb8-822fa92cd84d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:42.735981 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd68e4a-31f5-4d9c-bfb8-822fa92cd84d admin admin] PATH_INFO: `/v3/users/754c411f631449c09819c9e4fc06b0cb` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:42.737505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd68e4a-31f5-4d9c-bfb8-822fa92cd84d admin admin] RBAC: Authorizing `identity:delete_user(user_id=754c411f631449c09819c9e4fc06b0cb)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:42.739539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd68e4a-31f5-4d9c-bfb8-822fa92cd84d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:42.769883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:42.787179 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0cb56918-4417-4233-9deb-e93379cd00d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da60a417da4a4cbb94189abdbfa39d30', 'request_id': 'req-0cb56918-4417-4233-9deb-e93379cd00d7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:42.797966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0cb56918-4417-4233-9deb-e93379cd00d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da60a417da4a4cbb94189abdbfa39d30', 'request_id': 'req-0cb56918-4417-4233-9deb-e93379cd00d7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:42.800376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bfd68e4a-31f5-4d9c-bfb8-822fa92cd84d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '754c411f631449c09819c9e4fc06b0cb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:42.801060 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bfd68e4a-31f5-4d9c-bfb8-822fa92cd84d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '754c411f631449c09819c9e4fc06b0cb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:42.807050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0cb56918-4417-4233-9deb-e93379cd00d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da60a417da4a4cbb94189abdbfa39d30', 'request_id': 'req-0cb56918-4417-4233-9deb-e93379cd00d7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:42.811160 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bfd68e4a-31f5-4d9c-bfb8-822fa92cd84d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '754c411f631449c09819c9e4fc06b0cb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:42.811903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bfd68e4a-31f5-4d9c-bfb8-822fa92cd84d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '754c411f631449c09819c9e4fc06b0cb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:42.823138 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0cb56918-4417-4233-9deb-e93379cd00d7 admin admin] The token cache is being invalidate because project da60a417da4a4cbb94189abdbfa39d30 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:42.823625 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0cb56918-4417-4233-9deb-e93379cd00d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:42.825387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-0cb56918-4417-4233-9deb-e93379cd00d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:42.827428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 953/1901] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:42 2022] DELETE /identity/v3/projects/da60a417da4a4cbb94189abdbfa39d30 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:42.831150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb56918-4417-4233-9deb-e93379cd00d7 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:42.831619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-bfd68e4a-31f5-4d9c-bfb8-822fa92cd84d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '754c411f631449c09819c9e4fc06b0cb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:42.832778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0cb56918-4417-4233-9deb-e93379cd00d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:42.845906 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 949/1902] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:42 2022] DELETE /identity/v3/users/754c411f631449c09819c9e4fc06b0cb => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:42.850901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b20d3e-8d50-44c6-a55b-a529afb8435d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:42.851400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b20d3e-8d50-44c6-a55b-a529afb8435d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:42.851749 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b20d3e-8d50-44c6-a55b-a529afb8435d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:42.872504 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-73b20d3e-8d50-44c6-a55b-a529afb8435d None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:42.873840 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-73b20d3e-8d50-44c6-a55b-a529afb8435d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:42.918861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb56918-4417-4233-9deb-e93379cd00d7 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:42.950388 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 950/1903] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:42.959816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b20d3e-8d50-44c6-a55b-a529afb8435d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:42.961192 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-73b20d3e-8d50-44c6-a55b-a529afb8435d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:42.982026 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ede64-4219-45d9-991e-d3abf29913a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:42.983022 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ede64-4219-45d9-991e-d3abf29913a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:42.983311 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ede64-4219-45d9-991e-d3abf29913a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:42.983585 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9ede64-4219-45d9-991e-d3abf29913a8 admin admin] PATH_INFO: `/v3/users/f4f120d756614e1096353deefc9d5b7a` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:42.985390 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ede64-4219-45d9-991e-d3abf29913a8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f4f120d756614e1096353deefc9d5b7a)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:42.987319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9ede64-4219-45d9-991e-d3abf29913a8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:43.018787 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b20d3e-8d50-44c6-a55b-a529afb8435d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:43.022750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccafb6c-6f5c-4d1d-af65-ef5d2b7b237f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:43.023410 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccafb6c-6f5c-4d1d-af65-ef5d2b7b237f service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:43.023697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccafb6c-6f5c-4d1d-af65-ef5d2b7b237f service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:43.023979 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ccafb6c-6f5c-4d1d-af65-ef5d2b7b237f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:43.033183 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccafb6c-6f5c-4d1d-af65-ef5d2b7b237f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:43.034635 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ccafb6c-6f5c-4d1d-af65-ef5d2b7b237f service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:43.035780 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3e9ede64-4219-45d9-991e-d3abf29913a8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f4f120d756614e1096353deefc9d5b7a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.038725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:43.045541 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 951/1904] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:07:43.053915 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3e9ede64-4219-45d9-991e-d3abf29913a8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f4f120d756614e1096353deefc9d5b7a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.056882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:43.061912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3e9ede64-4219-45d9-991e-d3abf29913a8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f4f120d756614e1096353deefc9d5b7a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.063456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3e9ede64-4219-45d9-991e-d3abf29913a8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f4f120d756614e1096353deefc9d5b7a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.074441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3e9ede64-4219-45d9-991e-d3abf29913a8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f4f120d756614e1096353deefc9d5b7a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.081488 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 954/1905] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:42 2022] DELETE /identity/v3/users/f4f120d756614e1096353deefc9d5b7a => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:43.369698 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccafb6c-6f5c-4d1d-af65-ef5d2b7b237f service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:43.371373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4ccafb6c-6f5c-4d1d-af65-ef5d2b7b237f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:43.395334 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ccafb6c-6f5c-4d1d-af65-ef5d2b7b237f service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:43.467828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b2643-9532-4bef-9aeb-659a740b8d0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:43.469654 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b2643-9532-4bef-9aeb-659a740b8d0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:43.470047 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b2643-9532-4bef-9aeb-659a740b8d0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:43.470489 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317b2643-9532-4bef-9aeb-659a740b8d0e admin admin] PATH_INFO: `/v3/projects/96d67ee8493e495baeec2d70bf79224a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:43.472076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b2643-9532-4bef-9aeb-659a740b8d0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=96d67ee8493e495baeec2d70bf79224a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:43.474600 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317b2643-9532-4bef-9aeb-659a740b8d0e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:43.536489 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-317b2643-9532-4bef-9aeb-659a740b8d0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '96d67ee8493e495baeec2d70bf79224a', 'request_id': 'req-317b2643-9532-4bef-9aeb-659a740b8d0e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.543553 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-317b2643-9532-4bef-9aeb-659a740b8d0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '96d67ee8493e495baeec2d70bf79224a', 'request_id': 'req-317b2643-9532-4bef-9aeb-659a740b8d0e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.565686 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-317b2643-9532-4bef-9aeb-659a740b8d0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '96d67ee8493e495baeec2d70bf79224a', 'request_id': 'req-317b2643-9532-4bef-9aeb-659a740b8d0e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.585647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-317b2643-9532-4bef-9aeb-659a740b8d0e admin admin] The token cache is being invalidate because project 96d67ee8493e495baeec2d70bf79224a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:43.586103 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-317b2643-9532-4bef-9aeb-659a740b8d0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.586984 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-317b2643-9532-4bef-9aeb-659a740b8d0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.589314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 952/1906] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:43 2022] DELETE /identity/v3/projects/96d67ee8493e495baeec2d70bf79224a => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:43.593204 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ede64-4219-45d9-991e-d3abf29913a8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:43.595572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3e9ede64-4219-45d9-991e-d3abf29913a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:43.612208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b2643-9532-4bef-9aeb-659a740b8d0e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:43.614844 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-317b2643-9532-4bef-9aeb-659a740b8d0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:43.627442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9ede64-4219-45d9-991e-d3abf29913a8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:43.741665 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317b2643-9532-4bef-9aeb-659a740b8d0e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:43.747888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c427b905-0e4f-46dd-9299-943b89d471dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:43.749010 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427b905-0e4f-46dd-9299-943b89d471dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:43.749259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427b905-0e4f-46dd-9299-943b89d471dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:43.749653 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c427b905-0e4f-46dd-9299-943b89d471dd admin admin] PATH_INFO: `/v3/projects/befd285c3f78447b931814fbb57ee254` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:43.751792 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427b905-0e4f-46dd-9299-943b89d471dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=befd285c3f78447b931814fbb57ee254)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:43.752475 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc3e627-c54d-4b40-9e26-bf6aa871d10f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:43.753935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c427b905-0e4f-46dd-9299-943b89d471dd admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:43.759150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3e627-c54d-4b40-9e26-bf6aa871d10f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:43.759543 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3e627-c54d-4b40-9e26-bf6aa871d10f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:43.759714 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc3e627-c54d-4b40-9e26-bf6aa871d10f admin admin] PATH_INFO: `/v3/users/00f0195a2f6b47bdabd8c3b97838fa23` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:43.762992 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3e627-c54d-4b40-9e26-bf6aa871d10f admin admin] RBAC: Authorizing `identity:delete_user(user_id=00f0195a2f6b47bdabd8c3b97838fa23)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:43.769366 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc3e627-c54d-4b40-9e26-bf6aa871d10f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:43.841985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:43.850446 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7cc3e627-c54d-4b40-9e26-bf6aa871d10f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '00f0195a2f6b47bdabd8c3b97838fa23'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.857090 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c427b905-0e4f-46dd-9299-943b89d471dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'befd285c3f78447b931814fbb57ee254', 'request_id': 'req-c427b905-0e4f-46dd-9299-943b89d471dd'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.865966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7cc3e627-c54d-4b40-9e26-bf6aa871d10f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '00f0195a2f6b47bdabd8c3b97838fa23'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.866225 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c427b905-0e4f-46dd-9299-943b89d471dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'befd285c3f78447b931814fbb57ee254', 'request_id': 'req-c427b905-0e4f-46dd-9299-943b89d471dd'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.878754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7cc3e627-c54d-4b40-9e26-bf6aa871d10f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '00f0195a2f6b47bdabd8c3b97838fa23'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.879385 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7cc3e627-c54d-4b40-9e26-bf6aa871d10f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '00f0195a2f6b47bdabd8c3b97838fa23'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.882424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c427b905-0e4f-46dd-9299-943b89d471dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'befd285c3f78447b931814fbb57ee254', 'request_id': 'req-c427b905-0e4f-46dd-9299-943b89d471dd'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.886899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7cc3e627-c54d-4b40-9e26-bf6aa871d10f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '00f0195a2f6b47bdabd8c3b97838fa23'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.893394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 955/1907] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:43 2022] DELETE /identity/v3/users/00f0195a2f6b47bdabd8c3b97838fa23 => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:43.896035 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c427b905-0e4f-46dd-9299-943b89d471dd admin admin] The token cache is being invalidate because project befd285c3f78447b931814fbb57ee254 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:43.897616 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c427b905-0e4f-46dd-9299-943b89d471dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.898371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c427b905-0e4f-46dd-9299-943b89d471dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:43.900045 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 953/1908] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:43 2022] DELETE /identity/v3/projects/befd285c3f78447b931814fbb57ee254 => generated 0 bytes in 289 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:43.904851 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705a0cf-80f7-48b1-a753-e7cd59b9cc97 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:43.905145 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705a0cf-80f7-48b1-a753-e7cd59b9cc97 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:43.905428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7705a0cf-80f7-48b1-a753-e7cd59b9cc97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:43.927115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-7705a0cf-80f7-48b1-a753-e7cd59b9cc97 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:43.928336 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7705a0cf-80f7-48b1-a753-e7cd59b9cc97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:44.006268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 956/1909] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:44.008784 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aeafa3-68bf-4335-b8d7-19192f5a91eb None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:44.009107 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aeafa3-68bf-4335-b8d7-19192f5a91eb None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:44.009488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0aeafa3-68bf-4335-b8d7-19192f5a91eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:44.032365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f0aeafa3-68bf-4335-b8d7-19192f5a91eb None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:44.034124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f0aeafa3-68bf-4335-b8d7-19192f5a91eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:44.043425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 954/1910] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:44.047723 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705a0cf-80f7-48b1-a753-e7cd59b9cc97 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:44.056690 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:44.065607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7705a0cf-80f7-48b1-a753-e7cd59b9cc97 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:44.074569 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad0343-f8ab-44c7-97e6-584fa0e57c1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:44.075247 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad0343-f8ab-44c7-97e6-584fa0e57c1c admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:44.075563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad0343-f8ab-44c7-97e6-584fa0e57c1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:44.075848 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ad0343-f8ab-44c7-97e6-584fa0e57c1c admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:44.076803 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad0343-f8ab-44c7-97e6-584fa0e57c1c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:07:44.076962 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad0343-f8ab-44c7-97e6-584fa0e57c1c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:44.078797 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ad0343-f8ab-44c7-97e6-584fa0e57c1c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:44.094375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 957/1911] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:07:44.099880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aeafa3-68bf-4335-b8d7-19192f5a91eb None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:44.111039 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0aeafa3-68bf-4335-b8d7-19192f5a91eb None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:44.116673 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5c77b-4ce3-47a7-bfa1-3c57b5305b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:44.117351 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5c77b-4ce3-47a7-bfa1-3c57b5305b09 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:44.117643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5c77b-4ce3-47a7-bfa1-3c57b5305b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:44.117928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bca5c77b-4ce3-47a7-bfa1-3c57b5305b09 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:44.118681 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5c77b-4ce3-47a7-bfa1-3c57b5305b09 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:44.121014 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bca5c77b-4ce3-47a7-bfa1-3c57b5305b09 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:44.149949 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 955/1912] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:44 2022] POST /identity/v3/projects => generated 375 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:07:44.154443 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad0343-f8ab-44c7-97e6-584fa0e57c1c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:44.165635 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ad0343-f8ab-44c7-97e6-584fa0e57c1c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:44.218151 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5c77b-4ce3-47a7-bfa1-3c57b5305b09 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:44.220809 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bca5c77b-4ce3-47a7-bfa1-3c57b5305b09 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:44.225160 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1addca-9e21-43c3-a1ac-c1123e4afb4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:44.225845 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1addca-9e21-43c3-a1ac-c1123e4afb4e admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:44.226181 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1addca-9e21-43c3-a1ac-c1123e4afb4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:44.226567 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1addca-9e21-43c3-a1ac-c1123e4afb4e admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:44.227280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1addca-9e21-43c3-a1ac-c1123e4afb4e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:44.229370 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e1addca-9e21-43c3-a1ac-c1123e4afb4e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:44.236856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bca5c77b-4ce3-47a7-bfa1-3c57b5305b09 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:44.242332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bcfe624-aae4-48cd-a7d9-3af7789d665f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:44.243917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfe624-aae4-48cd-a7d9-3af7789d665f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:44.244211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfe624-aae4-48cd-a7d9-3af7789d665f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:44.244493 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bcfe624-aae4-48cd-a7d9-3af7789d665f admin admin] PATH_INFO: `/v3/projects/316691dd906f4272b392f532c00f4ac9` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:44.245873 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfe624-aae4-48cd-a7d9-3af7789d665f admin admin] RBAC: Authorizing `identity:delete_project(project_id=316691dd906f4272b392f532c00f4ac9)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:44.248099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bcfe624-aae4-48cd-a7d9-3af7789d665f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:44.253097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 958/1913] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:44 2022] POST /identity/v3/users => generated 288 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:07:44.257427 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1addca-9e21-43c3-a1ac-c1123e4afb4e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:44.269733 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e1addca-9e21-43c3-a1ac-c1123e4afb4e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:44.272533 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa2faf-98c3-49a6-8dc1-040b2f5accbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:44.273328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa2faf-98c3-49a6-8dc1-040b2f5accbe admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:44.273621 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa2faf-98c3-49a6-8dc1-040b2f5accbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:44.273905 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acfa2faf-98c3-49a6-8dc1-040b2f5accbe admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:44.274681 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa2faf-98c3-49a6-8dc1-040b2f5accbe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:44.276442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acfa2faf-98c3-49a6-8dc1-040b2f5accbe admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:44.290998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 959/1914] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:07:44.292177 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa2faf-98c3-49a6-8dc1-040b2f5accbe admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:44.301562 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2bcfe624-aae4-48cd-a7d9-3af7789d665f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '316691dd906f4272b392f532c00f4ac9', 'request_id': 'req-2bcfe624-aae4-48cd-a7d9-3af7789d665f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:44.303058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acfa2faf-98c3-49a6-8dc1-040b2f5accbe admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:44.307560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2bcfe624-aae4-48cd-a7d9-3af7789d665f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '316691dd906f4272b392f532c00f4ac9', 'request_id': 'req-2bcfe624-aae4-48cd-a7d9-3af7789d665f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:44.315696 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2bcfe624-aae4-48cd-a7d9-3af7789d665f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '316691dd906f4272b392f532c00f4ac9', 'request_id': 'req-2bcfe624-aae4-48cd-a7d9-3af7789d665f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:44.329229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2bcfe624-aae4-48cd-a7d9-3af7789d665f admin admin] The token cache is being invalidate because project 316691dd906f4272b392f532c00f4ac9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:44.329229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2bcfe624-aae4-48cd-a7d9-3af7789d665f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:44.329702 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2bcfe624-aae4-48cd-a7d9-3af7789d665f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:44.331548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 956/1915] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:44 2022] DELETE /identity/v3/projects/316691dd906f4272b392f532c00f4ac9 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:44.336043 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236c156-0bfe-407b-84cf-5a6ca3897024 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:44.336342 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236c156-0bfe-407b-84cf-5a6ca3897024 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:44.336626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a236c156-0bfe-407b-84cf-5a6ca3897024 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:44.355799 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a236c156-0bfe-407b-84cf-5a6ca3897024 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:44.357044 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a236c156-0bfe-407b-84cf-5a6ca3897024 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:44.370227 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac55c05-1aad-48ad-bdbd-4a8cb6894c62 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:44.370937 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac55c05-1aad-48ad-bdbd-4a8cb6894c62 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:44.371228 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac55c05-1aad-48ad-bdbd-4a8cb6894c62 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:44.371511 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac55c05-1aad-48ad-bdbd-4a8cb6894c62 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:44.372262 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac55c05-1aad-48ad-bdbd-4a8cb6894c62 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:44.373865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac55c05-1aad-48ad-bdbd-4a8cb6894c62 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:44.386344 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 960/1916] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:44.389188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac55c05-1aad-48ad-bdbd-4a8cb6894c62 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:44.391755 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-eac55c05-1aad-48ad-bdbd-4a8cb6894c62 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:44.404225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac55c05-1aad-48ad-bdbd-4a8cb6894c62 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:44.409550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c872667-f9e8-44e9-a1ac-3e27bb5d3fbe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:44.410464 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c872667-f9e8-44e9-a1ac-3e27bb5d3fbe admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:44.410739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c872667-f9e8-44e9-a1ac-3e27bb5d3fbe admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:44.411007 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c872667-f9e8-44e9-a1ac-3e27bb5d3fbe admin admin] PATH_INFO: `/v3/projects/9cab396f8bae4e8eb6863d861df82f4b/users/6184e7158e854caba48fd5e7ecfd76f9/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:44.426167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 957/1917] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:44.430340 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c872667-f9e8-44e9-a1ac-3e27bb5d3fbe admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cab396f8bae4e8eb6863d861df82f4b, user_id=6184e7158e854caba48fd5e7ecfd76f9, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:44.434719 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f28fed-47e6-456c-a454-28c50b6ed364 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:44.435017 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f28fed-47e6-456c-a454-28c50b6ed364 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:44.435130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c872667-f9e8-44e9-a1ac-3e27bb5d3fbe admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:44.435307 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f28fed-47e6-456c-a454-28c50b6ed364 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:44.459020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 961/1918] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:44 2022] PUT /identity/v3/projects/9cab396f8bae4e8eb6863d861df82f4b/users/6184e7158e854caba48fd5e7ecfd76f9/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:44.462443 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-12f28fed-47e6-456c-a454-28c50b6ed364 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:44.463679 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-12f28fed-47e6-456c-a454-28c50b6ed364 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:44.471004 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e86ef7-64cc-4c02-bddb-dbb94c0cad62 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:44.471279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e86ef7-64cc-4c02-bddb-dbb94c0cad62 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:44.471627 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e86ef7-64cc-4c02-bddb-dbb94c0cad62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:44.476010 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:44.504563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-f7e86ef7-64cc-4c02-bddb-dbb94c0cad62 None None] MFA Rules not processed for user `6184e7158e854caba48fd5e7ecfd76f9`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:44.506095 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f7e86ef7-64cc-4c02-bddb-dbb94c0cad62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:44.542786 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 958/1919] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:44.546998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:44.548152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f28fed-47e6-456c-a454-28c50b6ed364 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:44.560223 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f28fed-47e6-456c-a454-28c50b6ed364 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:44.567183 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae88fb-f87a-4b23-9e0b-2dd8465cd742 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:44.568244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae88fb-f87a-4b23-9e0b-2dd8465cd742 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:44.568842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae88fb-f87a-4b23-9e0b-2dd8465cd742 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:44.569200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae88fb-f87a-4b23-9e0b-2dd8465cd742 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:44.570031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae88fb-f87a-4b23-9e0b-2dd8465cd742 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:07:44.570469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae88fb-f87a-4b23-9e0b-2dd8465cd742 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:44.572882 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae88fb-f87a-4b23-9e0b-2dd8465cd742 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:44.587505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 959/1920] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:07:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 14:07:44.592056 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae88fb-f87a-4b23-9e0b-2dd8465cd742 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:44.598416 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 962/1921] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:44 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:44.602478 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e86ef7-64cc-4c02-bddb-dbb94c0cad62 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:44.604416 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae88fb-f87a-4b23-9e0b-2dd8465cd742 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:44.610581 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecf4c5-4a4e-45cb-a3fa-42ceeafbf25b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:44.611527 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecf4c5-4a4e-45cb-a3fa-42ceeafbf25b admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:44.611900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecf4c5-4a4e-45cb-a3fa-42ceeafbf25b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:44.612418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38ecf4c5-4a4e-45cb-a3fa-42ceeafbf25b admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:44.613746 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ecf4c5-4a4e-45cb-a3fa-42ceeafbf25b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:44.615543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38ecf4c5-4a4e-45cb-a3fa-42ceeafbf25b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:44.619785 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e86ef7-64cc-4c02-bddb-dbb94c0cad62 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:44.626012 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5186ed-f504-4ad5-ac42-0b1fa60f199c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:44.626796 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5186ed-f504-4ad5-ac42-0b1fa60f199c admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:44.627087 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5186ed-f504-4ad5-ac42-0b1fa60f199c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:44.627356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de5186ed-f504-4ad5-ac42-0b1fa60f199c admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:44.628062 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5186ed-f504-4ad5-ac42-0b1fa60f199c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:44.630324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de5186ed-f504-4ad5-ac42-0b1fa60f199c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:44.651159 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 960/1922] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:44 2022] POST /identity/v3/projects => generated 383 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:44.656542 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecf4c5-4a4e-45cb-a3fa-42ceeafbf25b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:44.665318 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 963/1923] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:44 2022] POST /identity/v3/projects => generated 375 bytes in 63 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:44.669036 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5186ed-f504-4ad5-ac42-0b1fa60f199c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:44.755244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38ecf4c5-4a4e-45cb-a3fa-42ceeafbf25b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:44.773507 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de5186ed-f504-4ad5-ac42-0b1fa60f199c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:44.779773 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae99fff-96f6-4418-989c-7ecf0ef5c730 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:44.780016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae99fff-96f6-4418-989c-7ecf0ef5c730 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:44.780308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae99fff-96f6-4418-989c-7ecf0ef5c730 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:44.780579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ae99fff-96f6-4418-989c-7ecf0ef5c730 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:44.781321 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae99fff-96f6-4418-989c-7ecf0ef5c730 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:44.783352 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ae99fff-96f6-4418-989c-7ecf0ef5c730 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:44.814021 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 964/1924] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:44 2022] POST /identity/v3/users => generated 294 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:07:44.825314 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae99fff-96f6-4418-989c-7ecf0ef5c730 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:44.838072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ae99fff-96f6-4418-989c-7ecf0ef5c730 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:44.850887 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a681983-a884-4ffb-84e2-827fcc390dd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:44.852024 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a681983-a884-4ffb-84e2-827fcc390dd1 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:44.852589 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ab395-9d87-4874-9328-0860287ff64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:44.852998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a681983-a884-4ffb-84e2-827fcc390dd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:44.853449 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ab395-9d87-4874-9328-0860287ff64a admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:44.853743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ab395-9d87-4874-9328-0860287ff64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:44.854024 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee7ab395-9d87-4874-9328-0860287ff64a admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:44.854546 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a681983-a884-4ffb-84e2-827fcc390dd1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:44.854773 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ab395-9d87-4874-9328-0860287ff64a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:44.856518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a681983-a884-4ffb-84e2-827fcc390dd1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:44.857049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee7ab395-9d87-4874-9328-0860287ff64a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:44.858396 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a681983-a884-4ffb-84e2-827fcc390dd1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:44.870689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 965/1925] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:44.875561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a681983-a884-4ffb-84e2-827fcc390dd1 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:44.891606 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 961/1926] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:44 2022] POST /identity/v3/users => generated 292 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:44.891948 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a681983-a884-4ffb-84e2-827fcc390dd1 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:44.895211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ab395-9d87-4874-9328-0860287ff64a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:44.898741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cff257-e837-4662-a83c-cff662ea5dc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:44.899815 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cff257-e837-4662-a83c-cff662ea5dc6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:44.900106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cff257-e837-4662-a83c-cff662ea5dc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:44.900532 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13cff257-e837-4662-a83c-cff662ea5dc6 admin admin] PATH_INFO: `/v3/projects/510dfe56e42b4f9f87719f50724ef592/users/199f581fc6a34d51b570178f8a71021e/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:44.906950 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee7ab395-9d87-4874-9328-0860287ff64a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:44.913364 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef99a9-fe63-49d1-9cd4-4ba2252c747c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:44.914049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef99a9-fe63-49d1-9cd4-4ba2252c747c admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:44.914377 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef99a9-fe63-49d1-9cd4-4ba2252c747c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:44.914644 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2eef99a9-fe63-49d1-9cd4-4ba2252c747c admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:44.915391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef99a9-fe63-49d1-9cd4-4ba2252c747c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:44.918559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2eef99a9-fe63-49d1-9cd4-4ba2252c747c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:44.926465 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cff257-e837-4662-a83c-cff662ea5dc6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=510dfe56e42b4f9f87719f50724ef592, user_id=199f581fc6a34d51b570178f8a71021e, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:44.929451 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13cff257-e837-4662-a83c-cff662ea5dc6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:44.938886 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 962/1927] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:07:44.943147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef99a9-fe63-49d1-9cd4-4ba2252c747c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:44.948678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 966/1928] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:44 2022] PUT /identity/v3/projects/510dfe56e42b4f9f87719f50724ef592/users/199f581fc6a34d51b570178f8a71021e/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 74 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:44.950149 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:44.953444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cff257-e837-4662-a83c-cff662ea5dc6 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:44.956204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2eef99a9-fe63-49d1-9cd4-4ba2252c747c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:45.025358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13cff257-e837-4662-a83c-cff662ea5dc6 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:45.029715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec49bd7-9a0a-4dfd-8e46-d19ea5f85f6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:45.029844 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567b979-5034-407a-8c28-c0406f720119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:45.030420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec49bd7-9a0a-4dfd-8e46-d19ea5f85f6f admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:45.030546 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567b979-5034-407a-8c28-c0406f720119 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:45.031227 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567b979-5034-407a-8c28-c0406f720119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:45.031353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec49bd7-9a0a-4dfd-8e46-d19ea5f85f6f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:45.031520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec49bd7-9a0a-4dfd-8e46-d19ea5f85f6f admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:45.031623 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f567b979-5034-407a-8c28-c0406f720119 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:45.032260 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567b979-5034-407a-8c28-c0406f720119 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:45.032368 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec49bd7-9a0a-4dfd-8e46-d19ea5f85f6f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:45.033853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f567b979-5034-407a-8c28-c0406f720119 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:45.033952 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cec49bd7-9a0a-4dfd-8e46-d19ea5f85f6f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:45.041144 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:45.044890 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 963/1929] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 103 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:07:45.045020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 967/1930] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:44 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:07:45.053731 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec49bd7-9a0a-4dfd-8e46-d19ea5f85f6f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:45.054324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567b979-5034-407a-8c28-c0406f720119 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:45.065107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cec49bd7-9a0a-4dfd-8e46-d19ea5f85f6f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:45.065272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f567b979-5034-407a-8c28-c0406f720119 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:45.069707 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7443bd61-deb6-45d0-b293-74f8a3d7aff4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:45.070026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77caec9-ae6c-4bbf-a4f4-5c188bc3dc56 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:45.070629 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443bd61-deb6-45d0-b293-74f8a3d7aff4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:45.070914 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443bd61-deb6-45d0-b293-74f8a3d7aff4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:45.071079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77caec9-ae6c-4bbf-a4f4-5c188bc3dc56 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:45.071186 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7443bd61-deb6-45d0-b293-74f8a3d7aff4 admin admin] PATH_INFO: `/v3/projects/dc47c527df8d433d81ed852bded631b1/users/cdda488e22944cdf8240a2262efa781f/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:45.071372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77caec9-ae6c-4bbf-a4f4-5c188bc3dc56 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:45.071652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77caec9-ae6c-4bbf-a4f4-5c188bc3dc56 admin admin] PATH_INFO: `/v3/projects/510dfe56e42b4f9f87719f50724ef592/users/199f581fc6a34d51b570178f8a71021e/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:45.074007 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77caec9-ae6c-4bbf-a4f4-5c188bc3dc56 admin admin] RBAC: Authorizing `identity:create_grant(project_id=510dfe56e42b4f9f87719f50724ef592, user_id=199f581fc6a34d51b570178f8a71021e, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:45.076655 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77caec9-ae6c-4bbf-a4f4-5c188bc3dc56 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:45.089365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 964/1931] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:45 2022] PUT /identity/v3/projects/510dfe56e42b4f9f87719f50724ef592/users/199f581fc6a34d51b570178f8a71021e/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:45.098882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443bd61-deb6-45d0-b293-74f8a3d7aff4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dc47c527df8d433d81ed852bded631b1, user_id=cdda488e22944cdf8240a2262efa781f, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:45.100307 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5e1ac-74ea-4fe9-b45b-b5613eb618fb None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:45.100604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5e1ac-74ea-4fe9-b45b-b5613eb618fb None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:45.100890 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24e5e1ac-74ea-4fe9-b45b-b5613eb618fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:45.101558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7443bd61-deb6-45d0-b293-74f8a3d7aff4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:45.115766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 968/1932] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:45 2022] PUT /identity/v3/projects/dc47c527df8d433d81ed852bded631b1/users/cdda488e22944cdf8240a2262efa781f/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:45.125471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487b296-fe99-451d-b7ea-8e14d450d0d0 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:45.125764 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487b296-fe99-451d-b7ea-8e14d450d0d0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:45.126045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5487b296-fe99-451d-b7ea-8e14d450d0d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:45.133347 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-24e5e1ac-74ea-4fe9-b45b-b5613eb618fb None None] MFA Rules not processed for user `199f581fc6a34d51b570178f8a71021e`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:45.133975 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-24e5e1ac-74ea-4fe9-b45b-b5613eb618fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:45.154746 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5487b296-fe99-451d-b7ea-8e14d450d0d0 None None] MFA Rules not processed for user `cdda488e22944cdf8240a2262efa781f`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:45.156528 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5487b296-fe99-451d-b7ea-8e14d450d0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:45.210186 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 965/1933] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:45.221257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5e1ac-74ea-4fe9-b45b-b5613eb618fb None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:45.222684 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-24e5e1ac-74ea-4fe9-b45b-b5613eb618fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:45.226836 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 969/1934] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:45 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:45.238656 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487b296-fe99-451d-b7ea-8e14d450d0d0 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:45.240093 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5487b296-fe99-451d-b7ea-8e14d450d0d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:45.292952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24e5e1ac-74ea-4fe9-b45b-b5613eb618fb None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:45.299906 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03873f13-b804-4b60-af21-17ec17b37aec service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:45.300575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03873f13-b804-4b60-af21-17ec17b37aec service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:45.300868 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03873f13-b804-4b60-af21-17ec17b37aec service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:45.301153 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03873f13-b804-4b60-af21-17ec17b37aec service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:45.308329 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5487b296-fe99-451d-b7ea-8e14d450d0d0 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:45.311938 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8fda9-6fc2-49b1-8f09-3ca325cdfa98 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:45.313113 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8fda9-6fc2-49b1-8f09-3ca325cdfa98 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:45.313397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8fda9-6fc2-49b1-8f09-3ca325cdfa98 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:45.313677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93d8fda9-6fc2-49b1-8f09-3ca325cdfa98 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:45.323981 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8fda9-6fc2-49b1-8f09-3ca325cdfa98 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:45.325535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93d8fda9-6fc2-49b1-8f09-3ca325cdfa98 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:45.331354 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:45.335486 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 970/1935] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:45 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:07:45.355000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8fda9-6fc2-49b1-8f09-3ca325cdfa98 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:45.371412 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03873f13-b804-4b60-af21-17ec17b37aec service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:45.372855 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03873f13-b804-4b60-af21-17ec17b37aec service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:45.383844 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 966/1936] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 162 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:07:45.419935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93d8fda9-6fc2-49b1-8f09-3ca325cdfa98 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:45.425591 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac1a65-c923-4bc8-b08f-393c606cfeed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:45.426532 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac1a65-c923-4bc8-b08f-393c606cfeed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:45.427208 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac1a65-c923-4bc8-b08f-393c606cfeed admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:45.427492 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eac1a65-c923-4bc8-b08f-393c606cfeed admin admin] PATH_INFO: `/v3/users/cdda488e22944cdf8240a2262efa781f` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:45.428799 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac1a65-c923-4bc8-b08f-393c606cfeed admin admin] RBAC: Authorizing `identity:delete_user(user_id=cdda488e22944cdf8240a2262efa781f)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:45.430760 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eac1a65-c923-4bc8-b08f-393c606cfeed admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:45.442784 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:45.482489 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1eac1a65-c923-4bc8-b08f-393c606cfeed admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cdda488e22944cdf8240a2262efa781f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:45.496866 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1eac1a65-c923-4bc8-b08f-393c606cfeed admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cdda488e22944cdf8240a2262efa781f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:45.504344 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1eac1a65-c923-4bc8-b08f-393c606cfeed admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cdda488e22944cdf8240a2262efa781f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:45.505016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1eac1a65-c923-4bc8-b08f-393c606cfeed admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cdda488e22944cdf8240a2262efa781f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:45.512192 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1eac1a65-c923-4bc8-b08f-393c606cfeed admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cdda488e22944cdf8240a2262efa781f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:45.519392 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 971/1937] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:45 2022] DELETE /identity/v3/users/cdda488e22944cdf8240a2262efa781f => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:45.524509 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9466cac-7c93-47a8-ab96-a79ae120bf37 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:45.525012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9466cac-7c93-47a8-ab96-a79ae120bf37 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:45.525427 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9466cac-7c93-47a8-ab96-a79ae120bf37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:45.545963 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a9466cac-7c93-47a8-ab96-a79ae120bf37 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:45.547772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a9466cac-7c93-47a8-ab96-a79ae120bf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:45.612304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 967/1938] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:45.622119 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac1a65-c923-4bc8-b08f-393c606cfeed admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:45.623594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1eac1a65-c923-4bc8-b08f-393c606cfeed admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:45.695811 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eac1a65-c923-4bc8-b08f-393c606cfeed admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:45.700011 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6daf2842-d9f3-4653-8820-0e415b1878cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:45.701010 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf2842-d9f3-4653-8820-0e415b1878cf service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:45.701010 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf2842-d9f3-4653-8820-0e415b1878cf service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:45.701258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6daf2842-d9f3-4653-8820-0e415b1878cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:45.712552 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf2842-d9f3-4653-8820-0e415b1878cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:45.714127 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6daf2842-d9f3-4653-8820-0e415b1878cf service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:45.726679 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 972/1939] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:46.221793 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9466cac-7c93-47a8-ab96-a79ae120bf37 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:46.223352 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a9466cac-7c93-47a8-ab96-a79ae120bf37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:46.235468 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9466cac-7c93-47a8-ab96-a79ae120bf37 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:46.241024 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00cd2c76-7958-421e-bb7c-f878632300b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:46.241903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd2c76-7958-421e-bb7c-f878632300b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:46.242397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd2c76-7958-421e-bb7c-f878632300b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:46.248143 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00cd2c76-7958-421e-bb7c-f878632300b2 admin admin] PATH_INFO: `/v3/projects/dc47c527df8d433d81ed852bded631b1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:46.250082 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd2c76-7958-421e-bb7c-f878632300b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dc47c527df8d433d81ed852bded631b1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:46.252847 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00cd2c76-7958-421e-bb7c-f878632300b2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:46.303613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-00cd2c76-7958-421e-bb7c-f878632300b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dc47c527df8d433d81ed852bded631b1', 'request_id': 'req-00cd2c76-7958-421e-bb7c-f878632300b2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:46.310667 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-00cd2c76-7958-421e-bb7c-f878632300b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dc47c527df8d433d81ed852bded631b1', 'request_id': 'req-00cd2c76-7958-421e-bb7c-f878632300b2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:46.318482 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-00cd2c76-7958-421e-bb7c-f878632300b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dc47c527df8d433d81ed852bded631b1', 'request_id': 'req-00cd2c76-7958-421e-bb7c-f878632300b2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:46.331101 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-00cd2c76-7958-421e-bb7c-f878632300b2 admin admin] The token cache is being invalidate because project dc47c527df8d433d81ed852bded631b1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:46.331393 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-00cd2c76-7958-421e-bb7c-f878632300b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:46.332031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-00cd2c76-7958-421e-bb7c-f878632300b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:46.333629 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 968/1940] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:46 2022] DELETE /identity/v3/projects/dc47c527df8d433d81ed852bded631b1 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:46.338515 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b78443-13de-468a-9e2b-a8145a71f5c2 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:46.338881 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b78443-13de-468a-9e2b-a8145a71f5c2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:46.339084 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76b78443-13de-468a-9e2b-a8145a71f5c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:46.357239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-76b78443-13de-468a-9e2b-a8145a71f5c2 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:46.358490 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-76b78443-13de-468a-9e2b-a8145a71f5c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:46.436133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 973/1941] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:46.445417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a988c-1c1d-451b-8a69-0fafe0967e2d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:46.445714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a988c-1c1d-451b-8a69-0fafe0967e2d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:46.445993 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f6a988c-1c1d-451b-8a69-0fafe0967e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:46.466516 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3f6a988c-1c1d-451b-8a69-0fafe0967e2d None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:46.467956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3f6a988c-1c1d-451b-8a69-0fafe0967e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:46.482485 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 969/1942] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:46.487071 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b78443-13de-468a-9e2b-a8145a71f5c2 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:46.496879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76b78443-13de-468a-9e2b-a8145a71f5c2 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:46.503369 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b91a5c-7d11-4f1e-a5fa-5463a2fb7d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:46.504051 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b91a5c-7d11-4f1e-a5fa-5463a2fb7d05 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:46.504371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b91a5c-7d11-4f1e-a5fa-5463a2fb7d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:46.504658 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18b91a5c-7d11-4f1e-a5fa-5463a2fb7d05 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:46.505434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b91a5c-7d11-4f1e-a5fa-5463a2fb7d05 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:07:46.505752 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b91a5c-7d11-4f1e-a5fa-5463a2fb7d05 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:46.507442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18b91a5c-7d11-4f1e-a5fa-5463a2fb7d05 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:46.521126 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 974/1943] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:07:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:07:46.525412 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a988c-1c1d-451b-8a69-0fafe0967e2d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:46.535504 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f6a988c-1c1d-451b-8a69-0fafe0967e2d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:46.540845 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee675cd5-5126-47d8-bc33-acacd36245d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:46.541541 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee675cd5-5126-47d8-bc33-acacd36245d7 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:46.541910 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee675cd5-5126-47d8-bc33-acacd36245d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:46.542199 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee675cd5-5126-47d8-bc33-acacd36245d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:46.542949 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee675cd5-5126-47d8-bc33-acacd36245d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:46.544880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee675cd5-5126-47d8-bc33-acacd36245d7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:46.572532 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 970/1944] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:46 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:46.577425 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b91a5c-7d11-4f1e-a5fa-5463a2fb7d05 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:46.588375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18b91a5c-7d11-4f1e-a5fa-5463a2fb7d05 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:46.601195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:46.646045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21614763-f12a-44eb-b987-ebf4fcd2da0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:46.646759 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21614763-f12a-44eb-b987-ebf4fcd2da0b admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:46.647051 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21614763-f12a-44eb-b987-ebf4fcd2da0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:46.647332 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21614763-f12a-44eb-b987-ebf4fcd2da0b admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:46.648031 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21614763-f12a-44eb-b987-ebf4fcd2da0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:46.649959 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21614763-f12a-44eb-b987-ebf4fcd2da0b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:46.670479 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 975/1945] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:46 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:46.674907 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee675cd5-5126-47d8-bc33-acacd36245d7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:46.686054 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee675cd5-5126-47d8-bc33-acacd36245d7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:46.692478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66606e-d708-445f-9c25-0d0a0ef69972 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:46.692690 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66606e-d708-445f-9c25-0d0a0ef69972 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:46.692969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66606e-d708-445f-9c25-0d0a0ef69972 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:46.693255 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc66606e-d708-445f-9c25-0d0a0ef69972 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:46.693983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66606e-d708-445f-9c25-0d0a0ef69972 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:46.695622 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc66606e-d708-445f-9c25-0d0a0ef69972 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:46.712196 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 971/1946] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:46.713321 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21614763-f12a-44eb-b987-ebf4fcd2da0b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:46.723997 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21614763-f12a-44eb-b987-ebf4fcd2da0b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:46.728849 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6040a78d-d033-4a07-a7f8-f012297b16e0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:46.729526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040a78d-d033-4a07-a7f8-f012297b16e0 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:46.729809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040a78d-d033-4a07-a7f8-f012297b16e0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:46.730093 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6040a78d-d033-4a07-a7f8-f012297b16e0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:46.730893 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6040a78d-d033-4a07-a7f8-f012297b16e0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:46.732472 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6040a78d-d033-4a07-a7f8-f012297b16e0 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:46.747395 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 976/1947] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:46.752018 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66606e-d708-445f-9c25-0d0a0ef69972 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:46.761930 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc66606e-d708-445f-9c25-0d0a0ef69972 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:46.766557 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850597b9-cc23-4931-943d-dfd8e786f7be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:46.767439 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850597b9-cc23-4931-943d-dfd8e786f7be admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:46.767741 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850597b9-cc23-4931-943d-dfd8e786f7be admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:46.768002 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850597b9-cc23-4931-943d-dfd8e786f7be admin admin] PATH_INFO: `/v3/projects/ddd381ae098742d8aab1ad3745a23a93/users/25d926bdf81842f693964897a632438e/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:46.776102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:46.784316 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850597b9-cc23-4931-943d-dfd8e786f7be admin admin] RBAC: Authorizing `identity:create_grant(project_id=ddd381ae098742d8aab1ad3745a23a93, user_id=25d926bdf81842f693964897a632438e, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:46.787075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850597b9-cc23-4931-943d-dfd8e786f7be admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:46.800583 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 972/1948] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:46 2022] PUT /identity/v3/projects/ddd381ae098742d8aab1ad3745a23a93/users/25d926bdf81842f693964897a632438e/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:46.811171 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88731e6e-4f3a-48a5-a911-9651d8705dfc None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:46.811468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88731e6e-4f3a-48a5-a911-9651d8705dfc None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:46.811755 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88731e6e-4f3a-48a5-a911-9651d8705dfc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:46.840885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-88731e6e-4f3a-48a5-a911-9651d8705dfc None None] MFA Rules not processed for user `25d926bdf81842f693964897a632438e`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:46.842121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-88731e6e-4f3a-48a5-a911-9651d8705dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:46.930431 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 977/1949] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:46.945481 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850597b9-cc23-4931-943d-dfd8e786f7be admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:46.947317 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-850597b9-cc23-4931-943d-dfd8e786f7be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:47.034655 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850597b9-cc23-4931-943d-dfd8e786f7be admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:47.039329 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b22db12-c4e2-4000-b013-514714f27735 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f80d59fec4994e689e857ba2087d3a23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:47.040176 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22db12-c4e2-4000-b013-514714f27735 service glance] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:47.040614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22db12-c4e2-4000-b013-514714f27735 service glance] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:47.041070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b22db12-c4e2-4000-b013-514714f27735 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:47.058530 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22db12-c4e2-4000-b013-514714f27735 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:47.060132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b22db12-c4e2-4000-b013-514714f27735 service glance] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:47.070958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 973/1950] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:48.979198 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88731e6e-4f3a-48a5-a911-9651d8705dfc None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:48.980899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-88731e6e-4f3a-48a5-a911-9651d8705dfc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:49.056508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88731e6e-4f3a-48a5-a911-9651d8705dfc None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:49.061904 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b41e06-c191-4076-8409-4ef2086b4b2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:49.062973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b41e06-c191-4076-8409-4ef2086b4b2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:49.063340 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b41e06-c191-4076-8409-4ef2086b4b2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:49.063668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94b41e06-c191-4076-8409-4ef2086b4b2c admin admin] PATH_INFO: `/v3/users/6184e7158e854caba48fd5e7ecfd76f9` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:49.065112 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b41e06-c191-4076-8409-4ef2086b4b2c admin admin] RBAC: Authorizing `identity:delete_user(user_id=6184e7158e854caba48fd5e7ecfd76f9)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:49.067282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94b41e06-c191-4076-8409-4ef2086b4b2c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:49.105673 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:49.125804 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-94b41e06-c191-4076-8409-4ef2086b4b2c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6184e7158e854caba48fd5e7ecfd76f9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:49.137770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-94b41e06-c191-4076-8409-4ef2086b4b2c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6184e7158e854caba48fd5e7ecfd76f9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:49.146770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-94b41e06-c191-4076-8409-4ef2086b4b2c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6184e7158e854caba48fd5e7ecfd76f9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:49.147468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-94b41e06-c191-4076-8409-4ef2086b4b2c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6184e7158e854caba48fd5e7ecfd76f9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:49.157508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-94b41e06-c191-4076-8409-4ef2086b4b2c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6184e7158e854caba48fd5e7ecfd76f9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:49.165660 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 978/1951] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:48 2022] DELETE /identity/v3/users/6184e7158e854caba48fd5e7ecfd76f9 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:49.170400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f213dba-5c40-40c1-a0da-c558abdd3d03 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:49.170943 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f213dba-5c40-40c1-a0da-c558abdd3d03 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:49.170943 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f213dba-5c40-40c1-a0da-c558abdd3d03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:49.190118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5f213dba-5c40-40c1-a0da-c558abdd3d03 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:49.190771 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5f213dba-5c40-40c1-a0da-c558abdd3d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:49.258087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 974/1952] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:49.269550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b41e06-c191-4076-8409-4ef2086b4b2c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:49.271246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-94b41e06-c191-4076-8409-4ef2086b4b2c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:49.337540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94b41e06-c191-4076-8409-4ef2086b4b2c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:49.341497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba392925-88a6-475b-904e-f8a0c560ef44 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:49.342399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba392925-88a6-475b-904e-f8a0c560ef44 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:49.342755 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba392925-88a6-475b-904e-f8a0c560ef44 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:49.343186 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba392925-88a6-475b-904e-f8a0c560ef44 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:49.353140 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba392925-88a6-475b-904e-f8a0c560ef44 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:49.354629 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba392925-88a6-475b-904e-f8a0c560ef44 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:49.366102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 979/1953] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:49.877674 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f213dba-5c40-40c1-a0da-c558abdd3d03 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:49.879951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5f213dba-5c40-40c1-a0da-c558abdd3d03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:49.896867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f213dba-5c40-40c1-a0da-c558abdd3d03 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:49.901269 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54316002-a347-47e9-acfc-bc506d80bb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:49.902149 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54316002-a347-47e9-acfc-bc506d80bb4d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:49.902499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54316002-a347-47e9-acfc-bc506d80bb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:49.902835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54316002-a347-47e9-acfc-bc506d80bb4d admin admin] PATH_INFO: `/v3/projects/9cab396f8bae4e8eb6863d861df82f4b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:49.903999 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54316002-a347-47e9-acfc-bc506d80bb4d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cab396f8bae4e8eb6863d861df82f4b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:49.908202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54316002-a347-47e9-acfc-bc506d80bb4d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:49.943021 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba392925-88a6-475b-904e-f8a0c560ef44 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:49.944784 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ba392925-88a6-475b-904e-f8a0c560ef44 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:49.973632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba392925-88a6-475b-904e-f8a0c560ef44 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:49.975055 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-54316002-a347-47e9-acfc-bc506d80bb4d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cab396f8bae4e8eb6863d861df82f4b', 'request_id': 'req-54316002-a347-47e9-acfc-bc506d80bb4d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:49.981821 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-54316002-a347-47e9-acfc-bc506d80bb4d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cab396f8bae4e8eb6863d861df82f4b', 'request_id': 'req-54316002-a347-47e9-acfc-bc506d80bb4d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:49.991570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-54316002-a347-47e9-acfc-bc506d80bb4d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cab396f8bae4e8eb6863d861df82f4b', 'request_id': 'req-54316002-a347-47e9-acfc-bc506d80bb4d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:49.994275 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:50.012826 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-54316002-a347-47e9-acfc-bc506d80bb4d admin admin] The token cache is being invalidate because project 9cab396f8bae4e8eb6863d861df82f4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:50.013088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-54316002-a347-47e9-acfc-bc506d80bb4d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:50.013843 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-54316002-a347-47e9-acfc-bc506d80bb4d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:50.018352 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 975/1954] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:49 2022] DELETE /identity/v3/projects/9cab396f8bae4e8eb6863d861df82f4b => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:50.020944 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54316002-a347-47e9-acfc-bc506d80bb4d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:50.022987 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-54316002-a347-47e9-acfc-bc506d80bb4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:50.059948 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e8e6806-7f5b-493d-b59b-27f138a0c96e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:50.060870 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e6806-7f5b-493d-b59b-27f138a0c96e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:50.061385 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e6806-7f5b-493d-b59b-27f138a0c96e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:50.061802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e8e6806-7f5b-493d-b59b-27f138a0c96e admin admin] PATH_INFO: `/v3/users/ce76587042e64fc2af6862d2af6d86f1` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:50.063616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e6806-7f5b-493d-b59b-27f138a0c96e admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce76587042e64fc2af6862d2af6d86f1)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:50.065782 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e8e6806-7f5b-493d-b59b-27f138a0c96e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:50.113301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54316002-a347-47e9-acfc-bc506d80bb4d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:50.119892 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c604e2-67d7-4df9-ab13-6b222d40b28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:50.122447 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c604e2-67d7-4df9-ab13-6b222d40b28d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:50.122784 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c604e2-67d7-4df9-ab13-6b222d40b28d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:50.123084 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c604e2-67d7-4df9-ab13-6b222d40b28d admin admin] PATH_INFO: `/v3/users/199f581fc6a34d51b570178f8a71021e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:50.124545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c604e2-67d7-4df9-ab13-6b222d40b28d admin admin] RBAC: Authorizing `identity:delete_user(user_id=199f581fc6a34d51b570178f8a71021e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:50.126798 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86c604e2-67d7-4df9-ab13-6b222d40b28d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:50.132047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4e8e6806-7f5b-493d-b59b-27f138a0c96e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce76587042e64fc2af6862d2af6d86f1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:50.135837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:50.157852 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4e8e6806-7f5b-493d-b59b-27f138a0c96e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce76587042e64fc2af6862d2af6d86f1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:50.167940 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4e8e6806-7f5b-493d-b59b-27f138a0c96e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce76587042e64fc2af6862d2af6d86f1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:50.168600 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4e8e6806-7f5b-493d-b59b-27f138a0c96e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce76587042e64fc2af6862d2af6d86f1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:50.180729 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4e8e6806-7f5b-493d-b59b-27f138a0c96e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce76587042e64fc2af6862d2af6d86f1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:50.189001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 980/1955] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:49 2022] DELETE /identity/v3/users/ce76587042e64fc2af6862d2af6d86f1 => generated 0 bytes in 246 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:50.193276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd53c4-1ee8-46f1-8c65-5e6124d4cebf None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:50.193566 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd53c4-1ee8-46f1-8c65-5e6124d4cebf None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:50.193846 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bbd53c4-1ee8-46f1-8c65-5e6124d4cebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:50.199474 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-86c604e2-67d7-4df9-ab13-6b222d40b28d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '199f581fc6a34d51b570178f8a71021e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:50.200327 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-86c604e2-67d7-4df9-ab13-6b222d40b28d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '199f581fc6a34d51b570178f8a71021e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:50.210746 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-86c604e2-67d7-4df9-ab13-6b222d40b28d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '199f581fc6a34d51b570178f8a71021e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:50.211849 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-86c604e2-67d7-4df9-ab13-6b222d40b28d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '199f581fc6a34d51b570178f8a71021e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:50.226555 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-86c604e2-67d7-4df9-ab13-6b222d40b28d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '199f581fc6a34d51b570178f8a71021e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:50.234878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4bbd53c4-1ee8-46f1-8c65-5e6124d4cebf None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:50.236593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4bbd53c4-1ee8-46f1-8c65-5e6124d4cebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:50.243447 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 976/1956] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:50 2022] DELETE /identity/v3/users/199f581fc6a34d51b570178f8a71021e => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:50.306662 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 981/1957] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:50.316605 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c604e2-67d7-4df9-ab13-6b222d40b28d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:50.318087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-86c604e2-67d7-4df9-ab13-6b222d40b28d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:50.383425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c604e2-67d7-4df9-ab13-6b222d40b28d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:50.387346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e0130-5d7a-491f-a73f-b6945baf6b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:50.388041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e0130-5d7a-491f-a73f-b6945baf6b7d service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:50.388332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e0130-5d7a-491f-a73f-b6945baf6b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:50.388616 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12e0130-5d7a-491f-a73f-b6945baf6b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:50.433866 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:50.476196 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e0130-5d7a-491f-a73f-b6945baf6b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:50.477793 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e12e0130-5d7a-491f-a73f-b6945baf6b7d service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:50.503381 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 977/1958] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:50.750391 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd53c4-1ee8-46f1-8c65-5e6124d4cebf None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:50.752577 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4bbd53c4-1ee8-46f1-8c65-5e6124d4cebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:50.782951 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bbd53c4-1ee8-46f1-8c65-5e6124d4cebf None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:50.798620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed74fd51-6ced-4655-ab4d-162d8daeb0d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:50.800312 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74fd51-6ced-4655-ab4d-162d8daeb0d7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:50.800598 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74fd51-6ced-4655-ab4d-162d8daeb0d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:50.800882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed74fd51-6ced-4655-ab4d-162d8daeb0d7 admin admin] PATH_INFO: `/v3/projects/510dfe56e42b4f9f87719f50724ef592` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:50.802406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74fd51-6ced-4655-ab4d-162d8daeb0d7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=510dfe56e42b4f9f87719f50724ef592)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:50.804341 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed74fd51-6ced-4655-ab4d-162d8daeb0d7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:50.898769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ed74fd51-6ced-4655-ab4d-162d8daeb0d7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '510dfe56e42b4f9f87719f50724ef592', 'request_id': 'req-ed74fd51-6ced-4655-ab4d-162d8daeb0d7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:50.926553 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ed74fd51-6ced-4655-ab4d-162d8daeb0d7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '510dfe56e42b4f9f87719f50724ef592', 'request_id': 'req-ed74fd51-6ced-4655-ab4d-162d8daeb0d7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:50.936536 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ed74fd51-6ced-4655-ab4d-162d8daeb0d7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '510dfe56e42b4f9f87719f50724ef592', 'request_id': 'req-ed74fd51-6ced-4655-ab4d-162d8daeb0d7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:50.955779 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e0130-5d7a-491f-a73f-b6945baf6b7d service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:50.957319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e12e0130-5d7a-491f-a73f-b6945baf6b7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:50.959615 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ed74fd51-6ced-4655-ab4d-162d8daeb0d7 admin admin] The token cache is being invalidate because project 510dfe56e42b4f9f87719f50724ef592 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:50.960122 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ed74fd51-6ced-4655-ab4d-162d8daeb0d7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:50.961040 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ed74fd51-6ced-4655-ab4d-162d8daeb0d7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:50.963190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 982/1959] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:50 2022] DELETE /identity/v3/projects/510dfe56e42b4f9f87719f50724ef592 => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:50.971880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb415ca8-2588-45ce-a9a6-ddb403a5e25e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:50.972961 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb415ca8-2588-45ce-a9a6-ddb403a5e25e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:50.973460 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb415ca8-2588-45ce-a9a6-ddb403a5e25e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:50.976703 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e12e0130-5d7a-491f-a73f-b6945baf6b7d service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:50.998491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-fb415ca8-2588-45ce-a9a6-ddb403a5e25e None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:51.000344 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-fb415ca8-2588-45ce-a9a6-ddb403a5e25e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:51.066339 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:51.076247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8ad1b15-3358-4d66-9336-efb7e03877e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:51.083414 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad1b15-3358-4d66-9336-efb7e03877e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:51.083414 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad1b15-3358-4d66-9336-efb7e03877e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:51.083414 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8ad1b15-3358-4d66-9336-efb7e03877e6 admin admin] PATH_INFO: `/v3/projects/5b52e9a898274997b56e86800aa4f04f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:51.083414 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad1b15-3358-4d66-9336-efb7e03877e6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5b52e9a898274997b56e86800aa4f04f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:51.084015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8ad1b15-3358-4d66-9336-efb7e03877e6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:51.095669 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 983/1960] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:51.103600 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e39d5-4f30-4bdc-93e8-9ddec4fb55d1 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:51.103797 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e39d5-4f30-4bdc-93e8-9ddec4fb55d1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:51.104079 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa9e39d5-4f30-4bdc-93e8-9ddec4fb55d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:51.125251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-fa9e39d5-4f30-4bdc-93e8-9ddec4fb55d1 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:51.126783 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-fa9e39d5-4f30-4bdc-93e8-9ddec4fb55d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:51.136131 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 984/1961] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:51.141940 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e39d5-4f30-4bdc-93e8-9ddec4fb55d1 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:51.151579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa9e39d5-4f30-4bdc-93e8-9ddec4fb55d1 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:51.155924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d8ad1b15-3358-4d66-9336-efb7e03877e6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5b52e9a898274997b56e86800aa4f04f', 'request_id': 'req-d8ad1b15-3358-4d66-9336-efb7e03877e6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:51.164780 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d8ad1b15-3358-4d66-9336-efb7e03877e6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5b52e9a898274997b56e86800aa4f04f', 'request_id': 'req-d8ad1b15-3358-4d66-9336-efb7e03877e6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:51.180360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d8ad1b15-3358-4d66-9336-efb7e03877e6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5b52e9a898274997b56e86800aa4f04f', 'request_id': 'req-d8ad1b15-3358-4d66-9336-efb7e03877e6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:51.196350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d8ad1b15-3358-4d66-9336-efb7e03877e6 admin admin] The token cache is being invalidate because project 5b52e9a898274997b56e86800aa4f04f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:51.196644 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d8ad1b15-3358-4d66-9336-efb7e03877e6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:51.197360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d8ad1b15-3358-4d66-9336-efb7e03877e6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:51.198989 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 978/1962] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:50 2022] DELETE /identity/v3/projects/5b52e9a898274997b56e86800aa4f04f => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:51.205519 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54386643-8df4-4199-a819-2ae013a3e12a None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:51.206214 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54386643-8df4-4199-a819-2ae013a3e12a None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:51.206214 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54386643-8df4-4199-a819-2ae013a3e12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:51.228785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-54386643-8df4-4199-a819-2ae013a3e12a None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:51.230040 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-54386643-8df4-4199-a819-2ae013a3e12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:51.233621 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923ecd2-6846-4131-a6d4-797803d47c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:51.234792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923ecd2-6846-4131-a6d4-797803d47c0f admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:51.235191 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923ecd2-6846-4131-a6d4-797803d47c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:51.235576 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e923ecd2-6846-4131-a6d4-797803d47c0f admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:51.236550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923ecd2-6846-4131-a6d4-797803d47c0f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:07:51.237134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923ecd2-6846-4131-a6d4-797803d47c0f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:51.239126 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e923ecd2-6846-4131-a6d4-797803d47c0f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:51.241005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 979/1963] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:51.248491 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1a03c-db1c-481b-9757-106327a34709 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:51.249046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1a03c-db1c-481b-9757-106327a34709 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:51.249793 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ef1a03c-db1c-481b-9757-106327a34709 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:51.260563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 985/1964] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 14:07:51.266147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923ecd2-6846-4131-a6d4-797803d47c0f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:51.272638 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e923ecd2-6846-4131-a6d4-797803d47c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:51.276361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0ef1a03c-db1c-481b-9757-106327a34709 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:51.279380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0ef1a03c-db1c-481b-9757-106327a34709 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:51.287507 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e923ecd2-6846-4131-a6d4-797803d47c0f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:51.292177 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 980/1965] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:51.293413 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41be68-050b-4897-970f-398b536689b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:51.294090 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41be68-050b-4897-970f-398b536689b0 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:51.294418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41be68-050b-4897-970f-398b536689b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:51.294699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e41be68-050b-4897-970f-398b536689b0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:51.295401 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41be68-050b-4897-970f-398b536689b0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:51.296257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1a03c-db1c-481b-9757-106327a34709 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:51.297417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e41be68-050b-4897-970f-398b536689b0 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:51.309124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ef1a03c-db1c-481b-9757-106327a34709 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:51.314834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae002a55-242c-44e4-81a3-2685abab4fe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:51.315798 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae002a55-242c-44e4-81a3-2685abab4fe1 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:51.316722 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae002a55-242c-44e4-81a3-2685abab4fe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:51.317119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae002a55-242c-44e4-81a3-2685abab4fe1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:51.318137 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae002a55-242c-44e4-81a3-2685abab4fe1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:07:51.318529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae002a55-242c-44e4-81a3-2685abab4fe1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:51.320340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae002a55-242c-44e4-81a3-2685abab4fe1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:51.336000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 981/1967] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:07:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 14:07:51.336268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 986/1967] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:51 2022] POST /identity/v3/projects => generated 377 bytes in 71 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:51.337001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae002a55-242c-44e4-81a3-2685abab4fe1 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:51.341001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41be68-050b-4897-970f-398b536689b0 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:51.349729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae002a55-242c-44e4-81a3-2685abab4fe1 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:51.381703 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:51.432062 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82297b-4afd-49dc-ae09-4dc013dff9cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:51.432866 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82297b-4afd-49dc-ae09-4dc013dff9cf admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:51.433234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82297b-4afd-49dc-ae09-4dc013dff9cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:51.433585 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f82297b-4afd-49dc-ae09-4dc013dff9cf admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:51.434402 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82297b-4afd-49dc-ae09-4dc013dff9cf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:51.437898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f82297b-4afd-49dc-ae09-4dc013dff9cf admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:51.441896 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e41be68-050b-4897-970f-398b536689b0 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:51.450224 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae61f1-e679-4592-b86d-9462bc2ba57b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:51.451114 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae61f1-e679-4592-b86d-9462bc2ba57b admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:51.451399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae61f1-e679-4592-b86d-9462bc2ba57b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:51.451684 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ae61f1-e679-4592-b86d-9462bc2ba57b admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:51.453198 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae61f1-e679-4592-b86d-9462bc2ba57b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:51.455238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ae61f1-e679-4592-b86d-9462bc2ba57b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:51.473932 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 982/1968] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:51 2022] POST /identity/v3/users => generated 289 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:51.486960 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82297b-4afd-49dc-ae09-4dc013dff9cf admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:51.500260 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f82297b-4afd-49dc-ae09-4dc013dff9cf admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:51.507678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcca69-d516-4f3a-9e23-3cd75c94da20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:51.509014 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcca69-d516-4f3a-9e23-3cd75c94da20 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:51.509014 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcca69-d516-4f3a-9e23-3cd75c94da20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:51.509234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cfcca69-d516-4f3a-9e23-3cd75c94da20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:51.509988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcca69-d516-4f3a-9e23-3cd75c94da20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:51.513854 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 987/1969] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:51 2022] POST /identity/v3/projects => generated 373 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:07:51.514048 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cfcca69-d516-4f3a-9e23-3cd75c94da20 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:51.515125 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae61f1-e679-4592-b86d-9462bc2ba57b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:51.527924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 983/1970] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:51.531823 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcca69-d516-4f3a-9e23-3cd75c94da20 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:51.553786 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cfcca69-d516-4f3a-9e23-3cd75c94da20 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:51.584778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:51.609082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ae61f1-e679-4592-b86d-9462bc2ba57b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:51.614254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df457f-4f12-4748-8491-270e528f350a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:51.614842 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df457f-4f12-4748-8491-270e528f350a admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:51.615134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df457f-4f12-4748-8491-270e528f350a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:51.615504 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61df457f-4f12-4748-8491-270e528f350a admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:51.616131 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61df457f-4f12-4748-8491-270e528f350a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:51.618112 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61df457f-4f12-4748-8491-270e528f350a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:51.626886 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67150d89-f695-4f9b-a5c8-b979379c72e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:51.628026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67150d89-f695-4f9b-a5c8-b979379c72e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:51.628317 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67150d89-f695-4f9b-a5c8-b979379c72e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:51.628601 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67150d89-f695-4f9b-a5c8-b979379c72e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:51.629641 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67150d89-f695-4f9b-a5c8-b979379c72e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:51.631471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67150d89-f695-4f9b-a5c8-b979379c72e4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:51.647793 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 988/1971] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:51 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:51.649780 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 984/1972] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:51.654655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df457f-4f12-4748-8491-270e528f350a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:51.656875 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67150d89-f695-4f9b-a5c8-b979379c72e4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:51.668047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61df457f-4f12-4748-8491-270e528f350a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:51.674494 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c257d65-ee39-4065-9d2d-c75ec0b43da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:51.675326 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c257d65-ee39-4065-9d2d-c75ec0b43da8 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:51.675623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c257d65-ee39-4065-9d2d-c75ec0b43da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:51.675981 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c257d65-ee39-4065-9d2d-c75ec0b43da8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:51.676724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c257d65-ee39-4065-9d2d-c75ec0b43da8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:51.678668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c257d65-ee39-4065-9d2d-c75ec0b43da8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:51.684591 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67150d89-f695-4f9b-a5c8-b979379c72e4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:51.690339 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 989/1973] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:51.694105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c257d65-ee39-4065-9d2d-c75ec0b43da8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:51.706820 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c257d65-ee39-4065-9d2d-c75ec0b43da8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:51.707198 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd37c6a1-c500-49af-993f-9ba25cdb5925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:51.708232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37c6a1-c500-49af-993f-9ba25cdb5925 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:51.708550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37c6a1-c500-49af-993f-9ba25cdb5925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:51.708861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd37c6a1-c500-49af-993f-9ba25cdb5925 admin admin] PATH_INFO: `/v3/projects/67b7c638788e4fc199878a03b4056114/users/64a38cbe5dd848a092023c84c1e3a5d8/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:51.713128 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679e2ab-4c62-4502-b31b-8994e24444aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:51.713817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679e2ab-4c62-4502-b31b-8994e24444aa admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:51.714101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679e2ab-4c62-4502-b31b-8994e24444aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:51.714417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1679e2ab-4c62-4502-b31b-8994e24444aa admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:51.715174 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679e2ab-4c62-4502-b31b-8994e24444aa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:51.716850 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1679e2ab-4c62-4502-b31b-8994e24444aa admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:51.732384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37c6a1-c500-49af-993f-9ba25cdb5925 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67b7c638788e4fc199878a03b4056114, user_id=64a38cbe5dd848a092023c84c1e3a5d8, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:51.735279 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd37c6a1-c500-49af-993f-9ba25cdb5925 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:51.736854 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 990/1974] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:51 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:51.741275 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679e2ab-4c62-4502-b31b-8994e24444aa admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:51.753020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1679e2ab-4c62-4502-b31b-8994e24444aa admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:51.754369 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 985/1975] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:51 2022] PUT /identity/v3/projects/67b7c638788e4fc199878a03b4056114/users/64a38cbe5dd848a092023c84c1e3a5d8/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:51.764583 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e71312-7263-4864-adb9-d6b1163398d4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:51.765541 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e71312-7263-4864-adb9-d6b1163398d4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:51.765853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8e71312-7263-4864-adb9-d6b1163398d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:51.798166 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b8e71312-7263-4864-adb9-d6b1163398d4 None None] MFA Rules not processed for user `64a38cbe5dd848a092023c84c1e3a5d8`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:51.799991 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b8e71312-7263-4864-adb9-d6b1163398d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:51.833993 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fac4b-de47-4ee0-9219-bbc31fe04dcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:51.834789 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fac4b-de47-4ee0-9219-bbc31fe04dcd admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:51.835081 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fac4b-de47-4ee0-9219-bbc31fe04dcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:51.835445 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8fac4b-de47-4ee0-9219-bbc31fe04dcd admin admin] PATH_INFO: `/v3/projects/dcebd5511b994d25a8b68e9cfe2294d3/users/8baa04ac294f4656a1aed1fa8d747161/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:51.856546 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8fac4b-de47-4ee0-9219-bbc31fe04dcd admin admin] RBAC: Authorizing `identity:create_grant(project_id=dcebd5511b994d25a8b68e9cfe2294d3, user_id=8baa04ac294f4656a1aed1fa8d747161, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:51.860206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8fac4b-de47-4ee0-9219-bbc31fe04dcd admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:51.877815 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 991/1976] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:51 2022] PUT /identity/v3/projects/dcebd5511b994d25a8b68e9cfe2294d3/users/8baa04ac294f4656a1aed1fa8d747161/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 137 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:51.882407 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 986/1977] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:51 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:51.886798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fac4b-de47-4ee0-9219-bbc31fe04dcd admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:51.891089 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bd5ee-8ef3-4efd-a629-181896575f08 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:51.891629 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bd5ee-8ef3-4efd-a629-181896575f08 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:51.892103 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421bd5ee-8ef3-4efd-a629-181896575f08 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:51.908224 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:51.931097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-421bd5ee-8ef3-4efd-a629-181896575f08 None None] MFA Rules not processed for user `8baa04ac294f4656a1aed1fa8d747161`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:51.933925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-421bd5ee-8ef3-4efd-a629-181896575f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:51.965711 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8fac4b-de47-4ee0-9219-bbc31fe04dcd admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:51.971255 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdceaf1b-61a5-4f61-b270-c3fff4b5d1df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:51.971934 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdceaf1b-61a5-4f61-b270-c3fff4b5d1df admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:51.972217 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdceaf1b-61a5-4f61-b270-c3fff4b5d1df admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:51.972491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdceaf1b-61a5-4f61-b270-c3fff4b5d1df admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:51.973742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdceaf1b-61a5-4f61-b270-c3fff4b5d1df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:51.975676 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdceaf1b-61a5-4f61-b270-c3fff4b5d1df admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:52.026922 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 992/1978] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:51 2022] POST /identity/v3/projects => generated 375 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:07:52.029150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdceaf1b-61a5-4f61-b270-c3fff4b5d1df admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:52.048278 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 987/1979] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:51 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 158 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:07:52.063614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421bd5ee-8ef3-4efd-a629-181896575f08 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:52.065392 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-421bd5ee-8ef3-4efd-a629-181896575f08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:52.067226 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:52.152580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdceaf1b-61a5-4f61-b270-c3fff4b5d1df admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:52.167673 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421bd5ee-8ef3-4efd-a629-181896575f08 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:52.169471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6a2ff-0286-42e7-bbe9-82f97bc1457e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:52.173312 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6a2ff-0286-42e7-bbe9-82f97bc1457e admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:52.173312 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6a2ff-0286-42e7-bbe9-82f97bc1457e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:52.173312 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dce6a2ff-0286-42e7-bbe9-82f97bc1457e admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:52.174153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce6a2ff-0286-42e7-bbe9-82f97bc1457e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:52.186679 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dce6a2ff-0286-42e7-bbe9-82f97bc1457e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:52.220024 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 993/1980] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:52 2022] POST /identity/v3/users => generated 288 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:52.227439 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6a2ff-0286-42e7-bbe9-82f97bc1457e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:52.230879 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c6f88-f1aa-4fbe-b6a1-b29ef44a30fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:52.231867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c6f88-f1aa-4fbe-b6a1-b29ef44a30fe service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:52.236377 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c6f88-f1aa-4fbe-b6a1-b29ef44a30fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:52.236377 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c6f88-f1aa-4fbe-b6a1-b29ef44a30fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:52.243314 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dce6a2ff-0286-42e7-bbe9-82f97bc1457e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:52.249573 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4da706-220f-4d3b-a315-38345c133e0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:52.250461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4da706-220f-4d3b-a315-38345c133e0a admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:52.250854 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4da706-220f-4d3b-a315-38345c133e0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:52.251266 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4da706-220f-4d3b-a315-38345c133e0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:52.252119 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4da706-220f-4d3b-a315-38345c133e0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:52.253868 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4da706-220f-4d3b-a315-38345c133e0a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:52.270036 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 994/1981] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:52.271993 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4da706-220f-4d3b-a315-38345c133e0a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:52.298143 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4da706-220f-4d3b-a315-38345c133e0a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:52.307335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3cd32d-5424-4647-94c8-cd637317cf66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:52.308254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3cd32d-5424-4647-94c8-cd637317cf66 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:52.308714 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3cd32d-5424-4647-94c8-cd637317cf66 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:52.309265 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b3cd32d-5424-4647-94c8-cd637317cf66 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:52.310289 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3cd32d-5424-4647-94c8-cd637317cf66 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:52.312200 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b3cd32d-5424-4647-94c8-cd637317cf66 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:52.320255 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c6f88-f1aa-4fbe-b6a1-b29ef44a30fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:52.321876 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-177c6f88-f1aa-4fbe-b6a1-b29ef44a30fe service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:52.324622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 995/1982] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 53 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:07:52.331530 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3cd32d-5424-4647-94c8-cd637317cf66 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:52.336243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 988/1983] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:52 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:52.346482 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b3cd32d-5424-4647-94c8-cd637317cf66 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:52.352705 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c24b9-822c-45cf-9b98-3465c5d6917b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:52.353801 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c24b9-822c-45cf-9b98-3465c5d6917b admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:52.354408 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c24b9-822c-45cf-9b98-3465c5d6917b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:52.354668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-377c24b9-822c-45cf-9b98-3465c5d6917b admin admin] PATH_INFO: `/v3/projects/ccf9acf2bdb848b39a149d9bc7ff1359/users/d6a063425b2341cf84af58bc66263ad2/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:52.381580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c24b9-822c-45cf-9b98-3465c5d6917b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccf9acf2bdb848b39a149d9bc7ff1359, user_id=d6a063425b2341cf84af58bc66263ad2, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:52.390849 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-377c24b9-822c-45cf-9b98-3465c5d6917b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:52.418626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 996/1984] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:52 2022] PUT /identity/v3/projects/ccf9acf2bdb848b39a149d9bc7ff1359/users/d6a063425b2341cf84af58bc66263ad2/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:52.438423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62c2fb-1237-4233-ad5d-de3f6410e7c6 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:52.438921 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62c2fb-1237-4233-ad5d-de3f6410e7c6 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:52.439430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd62c2fb-1237-4233-ad5d-de3f6410e7c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:52.458810 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:52.478511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-bd62c2fb-1237-4233-ad5d-de3f6410e7c6 None None] MFA Rules not processed for user `d6a063425b2341cf84af58bc66263ad2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:52.480073 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bd62c2fb-1237-4233-ad5d-de3f6410e7c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:52.606248 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 989/1985] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:52 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:52.613802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c24b9-822c-45cf-9b98-3465c5d6917b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:52.740677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-377c24b9-822c-45cf-9b98-3465c5d6917b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:52.757138 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15298b1f-eabc-4377-9088-6571b3fc1daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:52.757377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15298b1f-eabc-4377-9088-6571b3fc1daa admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:52.757590 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15298b1f-eabc-4377-9088-6571b3fc1daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:52.757867 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15298b1f-eabc-4377-9088-6571b3fc1daa admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:52.758597 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15298b1f-eabc-4377-9088-6571b3fc1daa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:52.760534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15298b1f-eabc-4377-9088-6571b3fc1daa admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:52.837152 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 997/1986] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:52 2022] POST /identity/v3/projects => generated 375 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:52.840995 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62c2fb-1237-4233-ad5d-de3f6410e7c6 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:52.855970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd62c2fb-1237-4233-ad5d-de3f6410e7c6 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:52.930716 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e1ad2-c4db-4594-81b9-6b1c4a0d0cc9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:52.931576 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e1ad2-c4db-4594-81b9-6b1c4a0d0cc9 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:52.931866 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e1ad2-c4db-4594-81b9-6b1c4a0d0cc9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:52.932141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6e1ad2-c4db-4594-81b9-6b1c4a0d0cc9 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:52.932889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e1ad2-c4db-4594-81b9-6b1c4a0d0cc9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:52.934822 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f6e1ad2-c4db-4594-81b9-6b1c4a0d0cc9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:52.956556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 990/1987] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:52 2022] POST /identity/v3/users => generated 294 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:52.960540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15298b1f-eabc-4377-9088-6571b3fc1daa admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:52.975542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15298b1f-eabc-4377-9088-6571b3fc1daa admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:52.981109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8658c-c60d-4f2d-bcab-8c1bcc1745a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:52.981818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8658c-c60d-4f2d-bcab-8c1bcc1745a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:52.982071 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8658c-c60d-4f2d-bcab-8c1bcc1745a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:52.982385 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d8658c-c60d-4f2d-bcab-8c1bcc1745a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:52.983125 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8658c-c60d-4f2d-bcab-8c1bcc1745a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:52.985413 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d8658c-c60d-4f2d-bcab-8c1bcc1745a3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:53.002532 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 998/1988] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:52 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:07:53.005362 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e1ad2-c4db-4594-81b9-6b1c4a0d0cc9 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:53.016516 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6e1ad2-c4db-4594-81b9-6b1c4a0d0cc9 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:53.021370 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6d078-bd77-4312-826f-3cce92ad2035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:53.022273 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6d078-bd77-4312-826f-3cce92ad2035 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:53.022628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6d078-bd77-4312-826f-3cce92ad2035 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:53.022910 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b6d078-bd77-4312-826f-3cce92ad2035 admin admin] PATH_INFO: `/v3/projects/375c8a2cb8944f688aeca334cbff0c45/users/fcbd7bed9689415cb967a38cf17b309b/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:53.038816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b6d078-bd77-4312-826f-3cce92ad2035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=375c8a2cb8944f688aeca334cbff0c45, user_id=fcbd7bed9689415cb967a38cf17b309b, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:53.041576 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b6d078-bd77-4312-826f-3cce92ad2035 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:53.055663 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 991/1989] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:53 2022] PUT /identity/v3/projects/375c8a2cb8944f688aeca334cbff0c45/users/fcbd7bed9689415cb967a38cf17b309b/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:53.060578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8658c-c60d-4f2d-bcab-8c1bcc1745a3 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:53.071371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d8658c-c60d-4f2d-bcab-8c1bcc1745a3 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:53.104492 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:53.130214 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4564e123-22ba-4807-a4f7-0a592ec50a73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:53.130940 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564e123-22ba-4807-a4f7-0a592ec50a73 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:53.131243 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564e123-22ba-4807-a4f7-0a592ec50a73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:53.131521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4564e123-22ba-4807-a4f7-0a592ec50a73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:53.132264 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564e123-22ba-4807-a4f7-0a592ec50a73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:53.133983 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4564e123-22ba-4807-a4f7-0a592ec50a73 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:53.144441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 999/1990] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:53 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:07:53.148499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6d078-bd77-4312-826f-3cce92ad2035 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:53.159608 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b6d078-bd77-4312-826f-3cce92ad2035 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:53.164803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84bb033-98f3-4ead-8e16-4f4b1ddad803 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:53.165714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bb033-98f3-4ead-8e16-4f4b1ddad803 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:53.166001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bb033-98f3-4ead-8e16-4f4b1ddad803 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:53.166277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84bb033-98f3-4ead-8e16-4f4b1ddad803 admin admin] PATH_INFO: `/v3/projects/375c8a2cb8944f688aeca334cbff0c45/users/fcbd7bed9689415cb967a38cf17b309b/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:53.168371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84bb033-98f3-4ead-8e16-4f4b1ddad803 admin admin] RBAC: Authorizing `identity:create_grant(project_id=375c8a2cb8944f688aeca334cbff0c45, user_id=fcbd7bed9689415cb967a38cf17b309b, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:53.171119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84bb033-98f3-4ead-8e16-4f4b1ddad803 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:53.182483 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 992/1991] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:53 2022] PUT /identity/v3/projects/375c8a2cb8944f688aeca334cbff0c45/users/fcbd7bed9689415cb967a38cf17b309b/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:53.194034 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dd42a-cb7b-427c-bc6c-094836545d55 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:53.194254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dd42a-cb7b-427c-bc6c-094836545d55 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:53.194570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f2dd42a-cb7b-427c-bc6c-094836545d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:53.221112 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5f2dd42a-cb7b-427c-bc6c-094836545d55 None None] MFA Rules not processed for user `fcbd7bed9689415cb967a38cf17b309b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:53.222425 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5f2dd42a-cb7b-427c-bc6c-094836545d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:53.297763 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1000/1992] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:53 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:53.309236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84bb033-98f3-4ead-8e16-4f4b1ddad803 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:53.311007 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e84bb033-98f3-4ead-8e16-4f4b1ddad803 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:53.379954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:53.387917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84bb033-98f3-4ead-8e16-4f4b1ddad803 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:53.392076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43b1e0-9bde-4e71-9555-b6303457e775 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:53.392844 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43b1e0-9bde-4e71-9555-b6303457e775 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:53.393240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43b1e0-9bde-4e71-9555-b6303457e775 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:53.393625 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce43b1e0-9bde-4e71-9555-b6303457e775 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:53.462290 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43b1e0-9bde-4e71-9555-b6303457e775 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:53.463873 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce43b1e0-9bde-4e71-9555-b6303457e775 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:53.473416 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 993/1993] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:53 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:55.023566 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dd42a-cb7b-427c-bc6c-094836545d55 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:55.033850 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f2dd42a-cb7b-427c-bc6c-094836545d55 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:55.037418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b51023-e877-433b-8d04-d2f32f18a931 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:55.038063 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b51023-e877-433b-8d04-d2f32f18a931 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:55.038387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b51023-e877-433b-8d04-d2f32f18a931 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:55.038673 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b51023-e877-433b-8d04-d2f32f18a931 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:55.119498 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b51023-e877-433b-8d04-d2f32f18a931 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:55.121207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b51023-e877-433b-8d04-d2f32f18a931 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:55.134401 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1001/1994] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:55 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:55.585267 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43b1e0-9bde-4e71-9555-b6303457e775 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:55.669347 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce43b1e0-9bde-4e71-9555-b6303457e775 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:55.674479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a0118f2-b305-46ac-9440-d5ff2e3afdf8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:55.675653 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0118f2-b305-46ac-9440-d5ff2e3afdf8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:55.675782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0118f2-b305-46ac-9440-d5ff2e3afdf8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:55.676058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a0118f2-b305-46ac-9440-d5ff2e3afdf8 admin admin] PATH_INFO: `/v3/users/8baa04ac294f4656a1aed1fa8d747161` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:55.677883 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0118f2-b305-46ac-9440-d5ff2e3afdf8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8baa04ac294f4656a1aed1fa8d747161)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:55.679868 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a0118f2-b305-46ac-9440-d5ff2e3afdf8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:55.733110 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7a0118f2-b305-46ac-9440-d5ff2e3afdf8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8baa04ac294f4656a1aed1fa8d747161'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:55.734413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7a0118f2-b305-46ac-9440-d5ff2e3afdf8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8baa04ac294f4656a1aed1fa8d747161'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:55.745440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7a0118f2-b305-46ac-9440-d5ff2e3afdf8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8baa04ac294f4656a1aed1fa8d747161'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:55.746112 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7a0118f2-b305-46ac-9440-d5ff2e3afdf8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8baa04ac294f4656a1aed1fa8d747161'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:55.763848 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7a0118f2-b305-46ac-9440-d5ff2e3afdf8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8baa04ac294f4656a1aed1fa8d747161'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:55.777270 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b51023-e877-433b-8d04-d2f32f18a931 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:55.783352 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 994/1995] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:07:55 2022] DELETE /identity/v3/users/8baa04ac294f4656a1aed1fa8d747161 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:07:55.786656 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b51023-e877-433b-8d04-d2f32f18a931 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:55.787535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5b201-88a6-4a3a-bdf3-025fda8e3a73 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:55.787905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5b201-88a6-4a3a-bdf3-025fda8e3a73 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:55.788251 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e5b201-88a6-4a3a-bdf3-025fda8e3a73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:55.807770 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-01e5b201-88a6-4a3a-bdf3-025fda8e3a73 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:55.809105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-01e5b201-88a6-4a3a-bdf3-025fda8e3a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:55.846004 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:55.861185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9dcb3d-1008-40fc-8162-6a2af5f13caf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:55.862456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9dcb3d-1008-40fc-8162-6a2af5f13caf service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:55.862860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9dcb3d-1008-40fc-8162-6a2af5f13caf service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:55.863289 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e9dcb3d-1008-40fc-8162-6a2af5f13caf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:55.865437 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0e9dcb3d-1008-40fc-8162-6a2af5f13caf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:55.880748 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:55.920182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 995/1996] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:55.929131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5b201-88a6-4a3a-bdf3-025fda8e3a73 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:55.931228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-01e5b201-88a6-4a3a-bdf3-025fda8e3a73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:55.961152 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9dcb3d-1008-40fc-8162-6a2af5f13caf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:55.963286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e9dcb3d-1008-40fc-8162-6a2af5f13caf service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:55.984320 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1002/1997] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:55 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:56.044972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e5b201-88a6-4a3a-bdf3-025fda8e3a73 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:56.049476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b194d9b-9a2e-4704-8a89-f2429acdec10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:56.050185 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b194d9b-9a2e-4704-8a89-f2429acdec10 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:56.050465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b194d9b-9a2e-4704-8a89-f2429acdec10 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:56.050747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b194d9b-9a2e-4704-8a89-f2429acdec10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:56.065249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b194d9b-9a2e-4704-8a89-f2429acdec10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:56.066644 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b194d9b-9a2e-4704-8a89-f2429acdec10 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:56.077823 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 996/1998] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 149 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:07:56.496371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9dcb3d-1008-40fc-8162-6a2af5f13caf service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:56.500579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0e9dcb3d-1008-40fc-8162-6a2af5f13caf service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:56.515239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e9dcb3d-1008-40fc-8162-6a2af5f13caf service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:56.520259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c72063-8dc5-4520-8218-c83b6f07d980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:56.521229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c72063-8dc5-4520-8218-c83b6f07d980 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:56.521712 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c72063-8dc5-4520-8218-c83b6f07d980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:56.522209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c72063-8dc5-4520-8218-c83b6f07d980 admin admin] PATH_INFO: `/v3/projects/dcebd5511b994d25a8b68e9cfe2294d3` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:56.523768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c72063-8dc5-4520-8218-c83b6f07d980 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dcebd5511b994d25a8b68e9cfe2294d3)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:56.529614 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c72063-8dc5-4520-8218-c83b6f07d980 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:56.590121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a0c72063-8dc5-4520-8218-c83b6f07d980 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dcebd5511b994d25a8b68e9cfe2294d3', 'request_id': 'req-a0c72063-8dc5-4520-8218-c83b6f07d980'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:56.611531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a0c72063-8dc5-4520-8218-c83b6f07d980 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dcebd5511b994d25a8b68e9cfe2294d3', 'request_id': 'req-a0c72063-8dc5-4520-8218-c83b6f07d980'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:56.611531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a0c72063-8dc5-4520-8218-c83b6f07d980 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dcebd5511b994d25a8b68e9cfe2294d3', 'request_id': 'req-a0c72063-8dc5-4520-8218-c83b6f07d980'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:56.624327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a0c72063-8dc5-4520-8218-c83b6f07d980 admin admin] The token cache is being invalidate because project dcebd5511b994d25a8b68e9cfe2294d3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:07:56.624999 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a0c72063-8dc5-4520-8218-c83b6f07d980 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:56.626363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a0c72063-8dc5-4520-8218-c83b6f07d980 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:07:56.628880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1003/1999] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:07:56 2022] DELETE /identity/v3/projects/dcebd5511b994d25a8b68e9cfe2294d3 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:56.633695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa768b01-7162-46b7-b51d-deb4d012b62d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:56.634210 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa768b01-7162-46b7-b51d-deb4d012b62d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:56.634341 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa768b01-7162-46b7-b51d-deb4d012b62d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:56.653996 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-fa768b01-7162-46b7-b51d-deb4d012b62d None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:56.656003 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fa768b01-7162-46b7-b51d-deb4d012b62d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:56.754066 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 997/2000] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:56.757494 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f1cbc-470a-4f79-8e71-8e272b6523e1 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:56.757790 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f1cbc-470a-4f79-8e71-8e272b6523e1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:56.758075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf9f1cbc-470a-4f79-8e71-8e272b6523e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:56.784113 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-bf9f1cbc-470a-4f79-8e71-8e272b6523e1 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:56.785371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bf9f1cbc-470a-4f79-8e71-8e272b6523e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:56.812274 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1004/2001] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:56.814890 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa768b01-7162-46b7-b51d-deb4d012b62d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:56.825928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa768b01-7162-46b7-b51d-deb4d012b62d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:56.839331 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669b488-1ea5-42fb-8b16-90dad71630a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:56.840082 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669b488-1ea5-42fb-8b16-90dad71630a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:56.840373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669b488-1ea5-42fb-8b16-90dad71630a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:56.840652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8669b488-1ea5-42fb-8b16-90dad71630a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:56.842039 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669b488-1ea5-42fb-8b16-90dad71630a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:07:56.842386 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669b488-1ea5-42fb-8b16-90dad71630a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:56.844193 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8669b488-1ea5-42fb-8b16-90dad71630a9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:56.857523 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 998/2002] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:07:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:07:56.862019 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f1cbc-470a-4f79-8e71-8e272b6523e1 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:56.877927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf9f1cbc-470a-4f79-8e71-8e272b6523e1 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:56.883400 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3e41e-ee58-4ee0-99ab-e167bac9e32d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:56.884091 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3e41e-ee58-4ee0-99ab-e167bac9e32d admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:56.884368 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3e41e-ee58-4ee0-99ab-e167bac9e32d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:56.884839 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48f3e41e-ee58-4ee0-99ab-e167bac9e32d admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:56.885372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f3e41e-ee58-4ee0-99ab-e167bac9e32d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:56.887357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48f3e41e-ee58-4ee0-99ab-e167bac9e32d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:56.936402 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1005/2003] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:07:56 2022] POST /identity/v3/projects => generated 369 bytes in 75 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:56.938287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669b488-1ea5-42fb-8b16-90dad71630a9 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:56.961314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8669b488-1ea5-42fb-8b16-90dad71630a9 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:57.047547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:57.060348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60a50f-9551-45fd-b4cd-357b7bfc9c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:57.061203 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60a50f-9551-45fd-b4cd-357b7bfc9c80 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:57.061635 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60a50f-9551-45fd-b4cd-357b7bfc9c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:57.062507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a60a50f-9551-45fd-b4cd-357b7bfc9c80 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:57.063474 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60a50f-9551-45fd-b4cd-357b7bfc9c80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:57.065576 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a60a50f-9551-45fd-b4cd-357b7bfc9c80 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:57.094009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 999/2004] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:07:56 2022] POST /identity/v3/users => generated 285 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:07:57.096576 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3e41e-ee58-4ee0-99ab-e167bac9e32d admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:57.115406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48f3e41e-ee58-4ee0-99ab-e167bac9e32d admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:57.124630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf84b32-3122-44c4-b8ff-33f8d18f44eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:57.126356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf84b32-3122-44c4-b8ff-33f8d18f44eb admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:57.129562 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf84b32-3122-44c4-b8ff-33f8d18f44eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:57.129562 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf84b32-3122-44c4-b8ff-33f8d18f44eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:57.129562 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf84b32-3122-44c4-b8ff-33f8d18f44eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:57.134020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf84b32-3122-44c4-b8ff-33f8d18f44eb admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:57.161754 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60a50f-9551-45fd-b4cd-357b7bfc9c80 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:57.168510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1006/2005] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 71 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:57.174857 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a60a50f-9551-45fd-b4cd-357b7bfc9c80 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:57.184170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3c74e8-e58a-4ed5-80b7-e79ee55760af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:57.185222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c74e8-e58a-4ed5-80b7-e79ee55760af admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:57.185621 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c74e8-e58a-4ed5-80b7-e79ee55760af admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:57.186156 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3c74e8-e58a-4ed5-80b7-e79ee55760af admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:57.189699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c74e8-e58a-4ed5-80b7-e79ee55760af admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:57.189699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3c74e8-e58a-4ed5-80b7-e79ee55760af admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:57.222193 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1000/2006] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:07:57 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:07:57.227370 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf84b32-3122-44c4-b8ff-33f8d18f44eb admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:57.244585 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf84b32-3122-44c4-b8ff-33f8d18f44eb admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:57.262828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9c99e-56ca-4356-9252-e38727067f97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:57.263760 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9c99e-56ca-4356-9252-e38727067f97 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:57.264045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9c99e-56ca-4356-9252-e38727067f97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:57.264429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3a9c99e-56ca-4356-9252-e38727067f97 admin admin] PATH_INFO: `/v3/projects/809cb51806b54475bc876aaa27c556c0/users/e1fcaa2414f44632b0ed99f7f150a013/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:57.325906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9c99e-56ca-4356-9252-e38727067f97 admin admin] RBAC: Authorizing `identity:create_grant(project_id=809cb51806b54475bc876aaa27c556c0, user_id=e1fcaa2414f44632b0ed99f7f150a013, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:57.329685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3a9c99e-56ca-4356-9252-e38727067f97 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:57.362934 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1007/2007] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:07:57 2022] PUT /identity/v3/projects/809cb51806b54475bc876aaa27c556c0/users/e1fcaa2414f44632b0ed99f7f150a013/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:07:57.373707 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778a788-b715-47f5-b200-401a042b3c01 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:57.374030 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778a788-b715-47f5-b200-401a042b3c01 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:57.375131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f778a788-b715-47f5-b200-401a042b3c01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:57.431026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f778a788-b715-47f5-b200-401a042b3c01 None None] MFA Rules not processed for user `e1fcaa2414f44632b0ed99f7f150a013`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:57.433035 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f778a788-b715-47f5-b200-401a042b3c01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:57.628848 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1001/2008] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:57 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 257 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:07:57.647538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9c99e-56ca-4356-9252-e38727067f97 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:07:57.650017 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e3a9c99e-56ca-4356-9252-e38727067f97 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:57.713485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:07:57.764416 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3a9c99e-56ca-4356-9252-e38727067f97 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:07:57.769561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a2304-ba96-45de-8437-b0a5cece44c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:07:57.770344 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a2304-ba96-45de-8437-b0a5cece44c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:57.770703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a2304-ba96-45de-8437-b0a5cece44c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:57.771052 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a2304-ba96-45de-8437-b0a5cece44c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:57.782862 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a2304-ba96-45de-8437-b0a5cece44c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:07:57.783627 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a2304-ba96-45de-8437-b0a5cece44c5 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:07:57.794016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1008/2009] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:07:57 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:07:59.302799 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506e9c3-794d-4534-8dc8-9761932c066b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:07:59.303331 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506e9c3-794d-4534-8dc8-9761932c066b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:07:59.303647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6506e9c3-794d-4534-8dc8-9761932c066b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:07:59.322790 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6506e9c3-794d-4534-8dc8-9761932c066b None None] MFA Rules not processed for user `e1fcaa2414f44632b0ed99f7f150a013`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:07:59.324306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6506e9c3-794d-4534-8dc8-9761932c066b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:07:59.336170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1002/2010] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:07:59 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:08:00.259776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a2304-ba96-45de-8437-b0a5cece44c5 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:00.263365 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-938a2304-ba96-45de-8437-b0a5cece44c5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:00.492025 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a2304-ba96-45de-8437-b0a5cece44c5 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:00.502217 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ad150-3478-4bb7-963c-886ca739a96f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:00.503376 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ad150-3478-4bb7-963c-886ca739a96f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:00.505296 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ad150-3478-4bb7-963c-886ca739a96f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:00.505686 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030ad150-3478-4bb7-963c-886ca739a96f admin admin] PATH_INFO: `/v3/users/64a38cbe5dd848a092023c84c1e3a5d8` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:00.510100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ad150-3478-4bb7-963c-886ca739a96f admin admin] RBAC: Authorizing `identity:delete_user(user_id=64a38cbe5dd848a092023c84c1e3a5d8)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:00.512155 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030ad150-3478-4bb7-963c-886ca739a96f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:00.653213 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-030ad150-3478-4bb7-963c-886ca739a96f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '64a38cbe5dd848a092023c84c1e3a5d8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:00.681279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-030ad150-3478-4bb7-963c-886ca739a96f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '64a38cbe5dd848a092023c84c1e3a5d8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:00.705003 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-030ad150-3478-4bb7-963c-886ca739a96f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '64a38cbe5dd848a092023c84c1e3a5d8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:00.705645 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-030ad150-3478-4bb7-963c-886ca739a96f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '64a38cbe5dd848a092023c84c1e3a5d8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:00.716011 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-030ad150-3478-4bb7-963c-886ca739a96f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '64a38cbe5dd848a092023c84c1e3a5d8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:00.727086 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1009/2011] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:08:00 2022] DELETE /identity/v3/users/64a38cbe5dd848a092023c84c1e3a5d8 => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:00.731952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85435ee4-e618-4d0e-b404-507c659ad159 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:00.734232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85435ee4-e618-4d0e-b404-507c659ad159 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:00.734708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85435ee4-e618-4d0e-b404-507c659ad159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:00.763535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-85435ee4-e618-4d0e-b404-507c659ad159 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:00.765841 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-85435ee4-e618-4d0e-b404-507c659ad159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:00.836890 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1003/2012] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:00.846716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ad150-3478-4bb7-963c-886ca739a96f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:00.848280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-030ad150-3478-4bb7-963c-886ca739a96f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:00.917502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:00.923959 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030ad150-3478-4bb7-963c-886ca739a96f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:00.928865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8f872-df80-4984-9e5c-0824ed6ad1d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:00.929736 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8f872-df80-4984-9e5c-0824ed6ad1d0 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:00.930192 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8f872-df80-4984-9e5c-0824ed6ad1d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:00.930717 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0c8f872-df80-4984-9e5c-0824ed6ad1d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:00.941163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8f872-df80-4984-9e5c-0824ed6ad1d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:00.942741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0c8f872-df80-4984-9e5c-0824ed6ad1d0 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:00.952789 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1010/2013] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:08:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:08:01.271839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85435ee4-e618-4d0e-b404-507c659ad159 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:01.273699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-85435ee4-e618-4d0e-b404-507c659ad159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:01.300112 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85435ee4-e618-4d0e-b404-507c659ad159 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:01.305761 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf145f-6804-4ffb-8c75-541a35a202bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:01.306935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf145f-6804-4ffb-8c75-541a35a202bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:01.307392 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf145f-6804-4ffb-8c75-541a35a202bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:01.308536 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfaf145f-6804-4ffb-8c75-541a35a202bc admin admin] PATH_INFO: `/v3/projects/67b7c638788e4fc199878a03b4056114` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:01.310155 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf145f-6804-4ffb-8c75-541a35a202bc admin admin] RBAC: Authorizing `identity:delete_project(project_id=67b7c638788e4fc199878a03b4056114)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:01.312703 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfaf145f-6804-4ffb-8c75-541a35a202bc admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:01.383184 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-dfaf145f-6804-4ffb-8c75-541a35a202bc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67b7c638788e4fc199878a03b4056114', 'request_id': 'req-dfaf145f-6804-4ffb-8c75-541a35a202bc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:01.384666 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:01.397443 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-dfaf145f-6804-4ffb-8c75-541a35a202bc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67b7c638788e4fc199878a03b4056114', 'request_id': 'req-dfaf145f-6804-4ffb-8c75-541a35a202bc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:01.412676 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-dfaf145f-6804-4ffb-8c75-541a35a202bc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67b7c638788e4fc199878a03b4056114', 'request_id': 'req-dfaf145f-6804-4ffb-8c75-541a35a202bc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:01.431079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-dfaf145f-6804-4ffb-8c75-541a35a202bc admin admin] The token cache is being invalidate because project 67b7c638788e4fc199878a03b4056114 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:08:01.431621 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-dfaf145f-6804-4ffb-8c75-541a35a202bc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:01.435496 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-dfaf145f-6804-4ffb-8c75-541a35a202bc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:01.440544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1004/2014] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:08:01 2022] DELETE /identity/v3/projects/67b7c638788e4fc199878a03b4056114 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:01.444359 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8f872-df80-4984-9e5c-0824ed6ad1d0 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:01.471462 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0c8f872-df80-4984-9e5c-0824ed6ad1d0 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:01.560757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf1de6-3ec6-4974-b24e-21ce785b1522 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:01.562321 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf1de6-3ec6-4974-b24e-21ce785b1522 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:01.563135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf1de6-3ec6-4974-b24e-21ce785b1522 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:01.563622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72cf1de6-3ec6-4974-b24e-21ce785b1522 admin admin] PATH_INFO: `/v3/users/d6a063425b2341cf84af58bc66263ad2` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:01.565095 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf1de6-3ec6-4974-b24e-21ce785b1522 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d6a063425b2341cf84af58bc66263ad2)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:01.571734 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72cf1de6-3ec6-4974-b24e-21ce785b1522 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:01.652880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-72cf1de6-3ec6-4974-b24e-21ce785b1522 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd6a063425b2341cf84af58bc66263ad2'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:01.669757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-72cf1de6-3ec6-4974-b24e-21ce785b1522 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd6a063425b2341cf84af58bc66263ad2'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:01.678785 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-72cf1de6-3ec6-4974-b24e-21ce785b1522 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd6a063425b2341cf84af58bc66263ad2'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:01.679444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-72cf1de6-3ec6-4974-b24e-21ce785b1522 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd6a063425b2341cf84af58bc66263ad2'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:01.688573 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-72cf1de6-3ec6-4974-b24e-21ce785b1522 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd6a063425b2341cf84af58bc66263ad2'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:01.699412 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1011/2015] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:08:01 2022] DELETE /identity/v3/users/d6a063425b2341cf84af58bc66263ad2 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:02.700471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf145f-6804-4ffb-8c75-541a35a202bc admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:02.703282 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dfaf145f-6804-4ffb-8c75-541a35a202bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:02.737418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfaf145f-6804-4ffb-8c75-541a35a202bc admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:02.807483 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aac235-7d81-41e5-99f3-3203991255ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:02.808370 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aac235-7d81-41e5-99f3-3203991255ac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:02.808781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aac235-7d81-41e5-99f3-3203991255ac admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:02.809254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57aac235-7d81-41e5-99f3-3203991255ac admin admin] PATH_INFO: `/v3/projects/ccf9acf2bdb848b39a149d9bc7ff1359` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:02.811715 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aac235-7d81-41e5-99f3-3203991255ac admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccf9acf2bdb848b39a149d9bc7ff1359)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:02.813801 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57aac235-7d81-41e5-99f3-3203991255ac admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:02.872511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-57aac235-7d81-41e5-99f3-3203991255ac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccf9acf2bdb848b39a149d9bc7ff1359', 'request_id': 'req-57aac235-7d81-41e5-99f3-3203991255ac'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:02.880924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-57aac235-7d81-41e5-99f3-3203991255ac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccf9acf2bdb848b39a149d9bc7ff1359', 'request_id': 'req-57aac235-7d81-41e5-99f3-3203991255ac'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:02.891301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-57aac235-7d81-41e5-99f3-3203991255ac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccf9acf2bdb848b39a149d9bc7ff1359', 'request_id': 'req-57aac235-7d81-41e5-99f3-3203991255ac'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:02.908832 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-57aac235-7d81-41e5-99f3-3203991255ac admin admin] The token cache is being invalidate because project ccf9acf2bdb848b39a149d9bc7ff1359 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:08:02.909127 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-57aac235-7d81-41e5-99f3-3203991255ac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:02.909721 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-57aac235-7d81-41e5-99f3-3203991255ac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:02.914594 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1005/2016] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:08:02 2022] DELETE /identity/v3/projects/ccf9acf2bdb848b39a149d9bc7ff1359 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:02.915574 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf1de6-3ec6-4974-b24e-21ce785b1522 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:02.917047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-72cf1de6-3ec6-4974-b24e-21ce785b1522 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:02.931360 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72cf1de6-3ec6-4974-b24e-21ce785b1522 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:03.009411 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d136ef2-0129-44c3-b32f-a939597bb337 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:03.010339 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d136ef2-0129-44c3-b32f-a939597bb337 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:03.010756 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d136ef2-0129-44c3-b32f-a939597bb337 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:03.011042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d136ef2-0129-44c3-b32f-a939597bb337 admin admin] PATH_INFO: `/v3/users/fcbd7bed9689415cb967a38cf17b309b` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:03.023237 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d136ef2-0129-44c3-b32f-a939597bb337 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fcbd7bed9689415cb967a38cf17b309b)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:03.025236 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d136ef2-0129-44c3-b32f-a939597bb337 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:03.044576 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:03.118259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7d136ef2-0129-44c3-b32f-a939597bb337 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fcbd7bed9689415cb967a38cf17b309b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:03.155889 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7d136ef2-0129-44c3-b32f-a939597bb337 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fcbd7bed9689415cb967a38cf17b309b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:03.174549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7d136ef2-0129-44c3-b32f-a939597bb337 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fcbd7bed9689415cb967a38cf17b309b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:03.184745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7d136ef2-0129-44c3-b32f-a939597bb337 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fcbd7bed9689415cb967a38cf17b309b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:03.193117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7d136ef2-0129-44c3-b32f-a939597bb337 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fcbd7bed9689415cb967a38cf17b309b'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:03.211107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1012/2017] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:08:02 2022] DELETE /identity/v3/users/fcbd7bed9689415cb967a38cf17b309b => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:03.612762 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aac235-7d81-41e5-99f3-3203991255ac admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:03.614468 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-57aac235-7d81-41e5-99f3-3203991255ac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:03.630424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57aac235-7d81-41e5-99f3-3203991255ac admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:03.716197 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09cb6552-7be6-4ab4-9c8c-de025f9c09e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:03.717142 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb6552-7be6-4ab4-9c8c-de025f9c09e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:03.717430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb6552-7be6-4ab4-9c8c-de025f9c09e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:03.717737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09cb6552-7be6-4ab4-9c8c-de025f9c09e9 admin admin] PATH_INFO: `/v3/projects/375c8a2cb8944f688aeca334cbff0c45` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:03.718981 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb6552-7be6-4ab4-9c8c-de025f9c09e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=375c8a2cb8944f688aeca334cbff0c45)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:03.720909 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09cb6552-7be6-4ab4-9c8c-de025f9c09e9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:03.731593 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:03.769831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-09cb6552-7be6-4ab4-9c8c-de025f9c09e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '375c8a2cb8944f688aeca334cbff0c45', 'request_id': 'req-09cb6552-7be6-4ab4-9c8c-de025f9c09e9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:03.775395 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-09cb6552-7be6-4ab4-9c8c-de025f9c09e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '375c8a2cb8944f688aeca334cbff0c45', 'request_id': 'req-09cb6552-7be6-4ab4-9c8c-de025f9c09e9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:03.782563 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-09cb6552-7be6-4ab4-9c8c-de025f9c09e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '375c8a2cb8944f688aeca334cbff0c45', 'request_id': 'req-09cb6552-7be6-4ab4-9c8c-de025f9c09e9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:03.795624 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-09cb6552-7be6-4ab4-9c8c-de025f9c09e9 admin admin] The token cache is being invalidate because project 375c8a2cb8944f688aeca334cbff0c45 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:08:03.796423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-09cb6552-7be6-4ab4-9c8c-de025f9c09e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:03.797085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-09cb6552-7be6-4ab4-9c8c-de025f9c09e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:03.798679 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1006/2018] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:08:03 2022] DELETE /identity/v3/projects/375c8a2cb8944f688aeca334cbff0c45 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:03.803754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ce61e-1b63-47b4-919a-ffa493e22cfa None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:03.804123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ce61e-1b63-47b4-919a-ffa493e22cfa None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:03.804471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867ce61e-1b63-47b4-919a-ffa493e22cfa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:03.824633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-867ce61e-1b63-47b4-919a-ffa493e22cfa None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:03.825947 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-867ce61e-1b63-47b4-919a-ffa493e22cfa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:03.888883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1013/2019] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:08:03.896950 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2955e7-0a64-406f-b2f1-04d082b48322 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:03.897153 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2955e7-0a64-406f-b2f1-04d082b48322 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:03.897425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2955e7-0a64-406f-b2f1-04d082b48322 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:03.918379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9c2955e7-0a64-406f-b2f1-04d082b48322 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:03.919618 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9c2955e7-0a64-406f-b2f1-04d082b48322 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:03.929053 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1007/2020] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:03.933197 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ce61e-1b63-47b4-919a-ffa493e22cfa None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:03.943966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867ce61e-1b63-47b4-919a-ffa493e22cfa None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:03.948660 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54345171-8fe3-4391-9b5d-b8abfe77ee99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:03.949335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54345171-8fe3-4391-9b5d-b8abfe77ee99 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:03.949619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54345171-8fe3-4391-9b5d-b8abfe77ee99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:03.949895 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54345171-8fe3-4391-9b5d-b8abfe77ee99 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:03.950674 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54345171-8fe3-4391-9b5d-b8abfe77ee99 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:08:03.950984 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54345171-8fe3-4391-9b5d-b8abfe77ee99 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:03.952631 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54345171-8fe3-4391-9b5d-b8abfe77ee99 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:03.966690 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1014/2021] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:08:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:08:03.971124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2955e7-0a64-406f-b2f1-04d082b48322 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:03.980766 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2955e7-0a64-406f-b2f1-04d082b48322 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:03.986175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab3c6ce-166c-4b29-a6d5-293a6d59bd90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:03.986888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3c6ce-166c-4b29-a6d5-293a6d59bd90 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:03.987179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3c6ce-166c-4b29-a6d5-293a6d59bd90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:03.987461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab3c6ce-166c-4b29-a6d5-293a6d59bd90 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:03.988181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab3c6ce-166c-4b29-a6d5-293a6d59bd90 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:03.990079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ab3c6ce-166c-4b29-a6d5-293a6d59bd90 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:04.031042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1008/2022] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:08:03 2022] POST /identity/v3/projects => generated 371 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:08:04.032207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54345171-8fe3-4391-9b5d-b8abfe77ee99 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:04.042685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54345171-8fe3-4391-9b5d-b8abfe77ee99 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:04.146008 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd37d0-5a5c-43e8-b7c5-42e1c5a4a06e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:04.146457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd37d0-5a5c-43e8-b7c5-42e1c5a4a06e admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:04.146916 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd37d0-5a5c-43e8-b7c5-42e1c5a4a06e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:04.147287 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cbd37d0-5a5c-43e8-b7c5-42e1c5a4a06e admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:04.147955 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd37d0-5a5c-43e8-b7c5-42e1c5a4a06e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:04.149900 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cbd37d0-5a5c-43e8-b7c5-42e1c5a4a06e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:04.174791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1015/2023] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:08:04 2022] POST /identity/v3/users => generated 286 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:08:04.179080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab3c6ce-166c-4b29-a6d5-293a6d59bd90 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:04.190393 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ab3c6ce-166c-4b29-a6d5-293a6d59bd90 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:04.195214 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a726bdf-7bc6-41cc-a848-9c70dcc0d660 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:04.195902 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a726bdf-7bc6-41cc-a848-9c70dcc0d660 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:04.196257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a726bdf-7bc6-41cc-a848-9c70dcc0d660 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:04.196546 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a726bdf-7bc6-41cc-a848-9c70dcc0d660 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:04.197307 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a726bdf-7bc6-41cc-a848-9c70dcc0d660 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:04.198971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a726bdf-7bc6-41cc-a848-9c70dcc0d660 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:04.211129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1009/2024] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:08:04.215314 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd37d0-5a5c-43e8-b7c5-42e1c5a4a06e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:04.223228 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:04.227637 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cbd37d0-5a5c-43e8-b7c5-42e1c5a4a06e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:04.234288 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85ad140-9de9-41a5-87c2-81fa1edee5d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:04.235066 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ad140-9de9-41a5-87c2-81fa1edee5d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:04.235700 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ad140-9de9-41a5-87c2-81fa1edee5d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:04.236060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85ad140-9de9-41a5-87c2-81fa1edee5d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:04.237168 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ad140-9de9-41a5-87c2-81fa1edee5d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:04.239094 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85ad140-9de9-41a5-87c2-81fa1edee5d5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:04.263008 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1016/2025] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:08:04 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:08:04.267061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a726bdf-7bc6-41cc-a848-9c70dcc0d660 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:04.278432 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a726bdf-7bc6-41cc-a848-9c70dcc0d660 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:04.283969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655725f0-1fa7-4cc4-8fed-e92e275d4322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:04.284880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655725f0-1fa7-4cc4-8fed-e92e275d4322 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:04.285171 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655725f0-1fa7-4cc4-8fed-e92e275d4322 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:04.285457 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-655725f0-1fa7-4cc4-8fed-e92e275d4322 admin admin] PATH_INFO: `/v3/projects/e3cc52e007fe407db072f8c788b984dd/users/db920784425247bfa592700df52ef5e2/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:04.303571 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655725f0-1fa7-4cc4-8fed-e92e275d4322 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3cc52e007fe407db072f8c788b984dd, user_id=db920784425247bfa592700df52ef5e2, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:04.306267 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-655725f0-1fa7-4cc4-8fed-e92e275d4322 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:04.334992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1010/2026] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:08:04 2022] PUT /identity/v3/projects/e3cc52e007fe407db072f8c788b984dd/users/db920784425247bfa592700df52ef5e2/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:04.338438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52a53f-b50e-4700-8257-2b81d1439d9b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:04.338738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52a53f-b50e-4700-8257-2b81d1439d9b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:04.339441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f52a53f-b50e-4700-8257-2b81d1439d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:04.375688 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8f52a53f-b50e-4700-8257-2b81d1439d9b None None] MFA Rules not processed for user `db920784425247bfa592700df52ef5e2`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:04.376967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8f52a53f-b50e-4700-8257-2b81d1439d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:04.479684 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1017/2027] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:04 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:04.491834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655725f0-1fa7-4cc4-8fed-e92e275d4322 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:04.493628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-655725f0-1fa7-4cc4-8fed-e92e275d4322 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:04.568633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-655725f0-1fa7-4cc4-8fed-e92e275d4322 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:04.574481 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3477c8b8-ec9b-4762-978f-8556c2d05b4e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f80d59fec4994e689e857ba2087d3a23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:04.575336 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477c8b8-ec9b-4762-978f-8556c2d05b4e service glance] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:04.575626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477c8b8-ec9b-4762-978f-8556c2d05b4e service glance] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:04.576233 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3477c8b8-ec9b-4762-978f-8556c2d05b4e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:04.590358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477c8b8-ec9b-4762-978f-8556c2d05b4e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:04.590358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3477c8b8-ec9b-4762-978f-8556c2d05b4e service glance] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:04.600384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1011/2028] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:08:04 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:08:14.476460 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=35 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:14.594892 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=73,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:15.605818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52a53f-b50e-4700-8257-2b81d1439d9b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:15.607390 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8f52a53f-b50e-4700-8257-2b81d1439d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:15.675055 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f52a53f-b50e-4700-8257-2b81d1439d9b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:15.679822 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775d66b-53ed-4087-bacd-975a1c630952 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:15.680733 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775d66b-53ed-4087-bacd-975a1c630952 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:15.681017 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775d66b-53ed-4087-bacd-975a1c630952 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:15.681296 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5775d66b-53ed-4087-bacd-975a1c630952 admin admin] PATH_INFO: `/v3/users/25d926bdf81842f693964897a632438e` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:15.682627 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775d66b-53ed-4087-bacd-975a1c630952 admin admin] RBAC: Authorizing `identity:delete_user(user_id=25d926bdf81842f693964897a632438e)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:15.684594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5775d66b-53ed-4087-bacd-975a1c630952 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:15.732696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-5775d66b-53ed-4087-bacd-975a1c630952 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '25d926bdf81842f693964897a632438e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:15.745649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-5775d66b-53ed-4087-bacd-975a1c630952 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '25d926bdf81842f693964897a632438e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:15.753879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-5775d66b-53ed-4087-bacd-975a1c630952 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '25d926bdf81842f693964897a632438e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:15.754562 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-5775d66b-53ed-4087-bacd-975a1c630952 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '25d926bdf81842f693964897a632438e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:15.762323 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-5775d66b-53ed-4087-bacd-975a1c630952 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '25d926bdf81842f693964897a632438e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:15.769068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1018/2029] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:08:15 2022] DELETE /identity/v3/users/25d926bdf81842f693964897a632438e => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:15.773906 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008ef2c-5b49-4130-a5ce-59d036adef56 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:15.774426 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008ef2c-5b49-4130-a5ce-59d036adef56 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:15.774614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e008ef2c-5b49-4130-a5ce-59d036adef56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:15.795316 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e008ef2c-5b49-4130-a5ce-59d036adef56 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:15.796514 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e008ef2c-5b49-4130-a5ce-59d036adef56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:15.859460 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1012/2030] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:15.870024 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775d66b-53ed-4087-bacd-975a1c630952 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:15.871565 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5775d66b-53ed-4087-bacd-975a1c630952 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:15.933264 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5775d66b-53ed-4087-bacd-975a1c630952 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:15.936970 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64892b6a-fadc-4e55-b54a-c384d0c4b0bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:15.937654 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64892b6a-fadc-4e55-b54a-c384d0c4b0bb service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:15.937944 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64892b6a-fadc-4e55-b54a-c384d0c4b0bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:15.938227 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64892b6a-fadc-4e55-b54a-c384d0c4b0bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:15.947671 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64892b6a-fadc-4e55-b54a-c384d0c4b0bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:15.949138 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64892b6a-fadc-4e55-b54a-c384d0c4b0bb service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:15.959104 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1019/2031] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:08:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:08:16.257860 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008ef2c-5b49-4130-a5ce-59d036adef56 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:16.259811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e008ef2c-5b49-4130-a5ce-59d036adef56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:16.278178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e008ef2c-5b49-4130-a5ce-59d036adef56 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:16.284217 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36d12a4-ac59-4f1c-b02f-3e04e2598927 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:16.285448 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d12a4-ac59-4f1c-b02f-3e04e2598927 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:16.285938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d12a4-ac59-4f1c-b02f-3e04e2598927 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:16.286639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36d12a4-ac59-4f1c-b02f-3e04e2598927 admin admin] PATH_INFO: `/v3/projects/ddd381ae098742d8aab1ad3745a23a93` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:16.288160 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d12a4-ac59-4f1c-b02f-3e04e2598927 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ddd381ae098742d8aab1ad3745a23a93)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:16.290373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36d12a4-ac59-4f1c-b02f-3e04e2598927 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:16.343842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e36d12a4-ac59-4f1c-b02f-3e04e2598927 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ddd381ae098742d8aab1ad3745a23a93', 'request_id': 'req-e36d12a4-ac59-4f1c-b02f-3e04e2598927'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:16.349500 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e36d12a4-ac59-4f1c-b02f-3e04e2598927 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ddd381ae098742d8aab1ad3745a23a93', 'request_id': 'req-e36d12a4-ac59-4f1c-b02f-3e04e2598927'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:16.356408 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e36d12a4-ac59-4f1c-b02f-3e04e2598927 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ddd381ae098742d8aab1ad3745a23a93', 'request_id': 'req-e36d12a4-ac59-4f1c-b02f-3e04e2598927'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:16.368889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e36d12a4-ac59-4f1c-b02f-3e04e2598927 admin admin] The token cache is being invalidate because project ddd381ae098742d8aab1ad3745a23a93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:08:16.369163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e36d12a4-ac59-4f1c-b02f-3e04e2598927 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:16.369921 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e36d12a4-ac59-4f1c-b02f-3e04e2598927 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:16.371575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1013/2032] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:08:16 2022] DELETE /identity/v3/projects/ddd381ae098742d8aab1ad3745a23a93 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:16.377283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd0e74-ce6e-4d34-9d45-07c182e7ef45 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:16.377580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd0e74-ce6e-4d34-9d45-07c182e7ef45 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:16.377860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cd0e74-ce6e-4d34-9d45-07c182e7ef45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:16.396818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c4cd0e74-ce6e-4d34-9d45-07c182e7ef45 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:16.398052 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c4cd0e74-ce6e-4d34-9d45-07c182e7ef45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:16.415718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:16.464820 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1020/2033] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:16.472242 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad8fe1-df28-4487-b150-d59ffa2e2ca3 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:16.472580 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad8fe1-df28-4487-b150-d59ffa2e2ca3 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:16.472794 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29ad8fe1-df28-4487-b150-d59ffa2e2ca3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:16.492894 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-29ad8fe1-df28-4487-b150-d59ffa2e2ca3 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:16.494111 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-29ad8fe1-df28-4487-b150-d59ffa2e2ca3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:16.515018 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1014/2034] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:16.519373 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd0e74-ce6e-4d34-9d45-07c182e7ef45 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:16.530654 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4cd0e74-ce6e-4d34-9d45-07c182e7ef45 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:16.536277 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd111393-c480-4397-a722-11eff5387981 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:16.537302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd111393-c480-4397-a722-11eff5387981 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:16.537595 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd111393-c480-4397-a722-11eff5387981 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:16.537962 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd111393-c480-4397-a722-11eff5387981 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:16.538840 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd111393-c480-4397-a722-11eff5387981 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:08:16.539259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd111393-c480-4397-a722-11eff5387981 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:16.541226 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd111393-c480-4397-a722-11eff5387981 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:16.562899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1021/2035] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:08:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:08:16.568052 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad8fe1-df28-4487-b150-d59ffa2e2ca3 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:16.581642 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29ad8fe1-df28-4487-b150-d59ffa2e2ca3 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:16.588275 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcc9a6-5665-4293-841c-42b65c186020 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:16.589151 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcc9a6-5665-4293-841c-42b65c186020 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:16.589540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcc9a6-5665-4293-841c-42b65c186020 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:16.589890 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fcc9a6-5665-4293-841c-42b65c186020 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:16.590732 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcc9a6-5665-4293-841c-42b65c186020 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:16.592710 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fcc9a6-5665-4293-841c-42b65c186020 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:16.620348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1015/2036] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:08:16 2022] POST /identity/v3/projects => generated 385 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:08:16.624208 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd111393-c480-4397-a722-11eff5387981 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:16.635556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd111393-c480-4397-a722-11eff5387981 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:16.692033 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762fe8b-9801-4f00-a5f1-406f017c4212 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:16.692713 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762fe8b-9801-4f00-a5f1-406f017c4212 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:16.693003 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762fe8b-9801-4f00-a5f1-406f017c4212 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:16.693284 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6762fe8b-9801-4f00-a5f1-406f017c4212 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:16.693996 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762fe8b-9801-4f00-a5f1-406f017c4212 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:16.696037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6762fe8b-9801-4f00-a5f1-406f017c4212 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:16.718218 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1022/2037] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:08:16 2022] POST /identity/v3/users => generated 293 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:08:16.722718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcc9a6-5665-4293-841c-42b65c186020 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:16.733310 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fcc9a6-5665-4293-841c-42b65c186020 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:16.738616 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97cef4-d679-4182-864a-adde3f28456b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:16.739275 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97cef4-d679-4182-864a-adde3f28456b admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:16.739558 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97cef4-d679-4182-864a-adde3f28456b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:16.739830 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d97cef4-d679-4182-864a-adde3f28456b admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:16.740555 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97cef4-d679-4182-864a-adde3f28456b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:16.742132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d97cef4-d679-4182-864a-adde3f28456b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:16.753216 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1016/2038] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:08:16.757212 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762fe8b-9801-4f00-a5f1-406f017c4212 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:16.767793 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6762fe8b-9801-4f00-a5f1-406f017c4212 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:16.773103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4295d483-e04b-4bd8-89ca-0eee849e5b12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:16.773780 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295d483-e04b-4bd8-89ca-0eee849e5b12 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:16.774056 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295d483-e04b-4bd8-89ca-0eee849e5b12 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:16.774367 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4295d483-e04b-4bd8-89ca-0eee849e5b12 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:16.775101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295d483-e04b-4bd8-89ca-0eee849e5b12 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:16.776695 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4295d483-e04b-4bd8-89ca-0eee849e5b12 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:16.805789 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1023/2039] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:08:16 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:08:16.807025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97cef4-d679-4182-864a-adde3f28456b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:16.817683 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d97cef4-d679-4182-864a-adde3f28456b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:16.823143 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebb8e3-d334-40f6-aa01-a298dcbf80ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:16.824036 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebb8e3-d334-40f6-aa01-a298dcbf80ea admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:16.824322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebb8e3-d334-40f6-aa01-a298dcbf80ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:16.824598 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ebb8e3-d334-40f6-aa01-a298dcbf80ea admin admin] PATH_INFO: `/v3/projects/9c84e3e932ad4cbeb1a750d2ed2615b8/users/b7155e000af34933b3577e29664e2c69/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:16.840327 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebb8e3-d334-40f6-aa01-a298dcbf80ea admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c84e3e932ad4cbeb1a750d2ed2615b8, user_id=b7155e000af34933b3577e29664e2c69, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:16.843035 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ebb8e3-d334-40f6-aa01-a298dcbf80ea admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:16.863179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1017/2040] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:08:16 2022] PUT /identity/v3/projects/9c84e3e932ad4cbeb1a750d2ed2615b8/users/b7155e000af34933b3577e29664e2c69/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:16.873700 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ff9af-e3c3-4ebb-841c-ecbaaae890e2 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:16.873910 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ff9af-e3c3-4ebb-841c-ecbaaae890e2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:16.874180 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-391ff9af-e3c3-4ebb-841c-ecbaaae890e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:16.903797 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-391ff9af-e3c3-4ebb-841c-ecbaaae890e2 None None] MFA Rules not processed for user `b7155e000af34933b3577e29664e2c69`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:16.905171 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-391ff9af-e3c3-4ebb-841c-ecbaaae890e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:16.973561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1024/2041] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:16 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:16.984559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebb8e3-d334-40f6-aa01-a298dcbf80ea admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:16.986861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d0ebb8e3-d334-40f6-aa01-a298dcbf80ea admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:17.014724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:17.062114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ebb8e3-d334-40f6-aa01-a298dcbf80ea admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:17.066529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-388f6eaa-d854-4bd9-85e7-feba20b0c839 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:17.067276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f6eaa-d854-4bd9-85e7-feba20b0c839 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:17.067566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f6eaa-d854-4bd9-85e7-feba20b0c839 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:17.067853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-388f6eaa-d854-4bd9-85e7-feba20b0c839 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:17.076996 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f6eaa-d854-4bd9-85e7-feba20b0c839 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:17.078514 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-388f6eaa-d854-4bd9-85e7-feba20b0c839 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:17.088097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1018/2042] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:08:16 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:08:19.645134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ff9af-e3c3-4ebb-841c-ecbaaae890e2 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:19.664315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:19.712995 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-391ff9af-e3c3-4ebb-841c-ecbaaae890e2 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:19.718798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7faefa6-2bd4-4e57-97d4-d6927a4970b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:19.719679 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7faefa6-2bd4-4e57-97d4-d6927a4970b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:19.719989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7faefa6-2bd4-4e57-97d4-d6927a4970b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:19.720262 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7faefa6-2bd4-4e57-97d4-d6927a4970b0 admin admin] PATH_INFO: `/v3/users/b7155e000af34933b3577e29664e2c69` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:19.721498 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7faefa6-2bd4-4e57-97d4-d6927a4970b0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b7155e000af34933b3577e29664e2c69)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:19.723531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7faefa6-2bd4-4e57-97d4-d6927a4970b0 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:19.782950 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f7faefa6-2bd4-4e57-97d4-d6927a4970b0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b7155e000af34933b3577e29664e2c69'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:19.797260 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f7faefa6-2bd4-4e57-97d4-d6927a4970b0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b7155e000af34933b3577e29664e2c69'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:19.805762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f7faefa6-2bd4-4e57-97d4-d6927a4970b0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b7155e000af34933b3577e29664e2c69'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:19.806543 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f7faefa6-2bd4-4e57-97d4-d6927a4970b0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b7155e000af34933b3577e29664e2c69'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:19.814903 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f7faefa6-2bd4-4e57-97d4-d6927a4970b0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b7155e000af34933b3577e29664e2c69'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:19.821347 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1025/2043] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:08:19 2022] DELETE /identity/v3/users/b7155e000af34933b3577e29664e2c69 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:19.826275 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e1c8a-939d-4bcc-8302-5ac8fa1bcbc2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:19.826782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e1c8a-939d-4bcc-8302-5ac8fa1bcbc2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:19.827063 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36e1c8a-939d-4bcc-8302-5ac8fa1bcbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:19.847057 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b36e1c8a-939d-4bcc-8302-5ac8fa1bcbc2 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:19.848274 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b36e1c8a-939d-4bcc-8302-5ac8fa1bcbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:19.915424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1019/2044] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:19.925823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7faefa6-2bd4-4e57-97d4-d6927a4970b0 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:19.927640 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f7faefa6-2bd4-4e57-97d4-d6927a4970b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:19.988354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7faefa6-2bd4-4e57-97d4-d6927a4970b0 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:19.992258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace91997-4148-4234-a56e-c7d50512cf51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:19.993078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace91997-4148-4234-a56e-c7d50512cf51 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:19.993248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace91997-4148-4234-a56e-c7d50512cf51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:19.993512 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace91997-4148-4234-a56e-c7d50512cf51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:20.004078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace91997-4148-4234-a56e-c7d50512cf51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:20.005539 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace91997-4148-4234-a56e-c7d50512cf51 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:20.015410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1026/2045] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:08:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:08:20.311432 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e1c8a-939d-4bcc-8302-5ac8fa1bcbc2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:20.313253 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b36e1c8a-939d-4bcc-8302-5ac8fa1bcbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:20.325905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36e1c8a-939d-4bcc-8302-5ac8fa1bcbc2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:20.331062 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682e566e-1a84-4f59-a76d-e45d4bacf264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:20.331938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e566e-1a84-4f59-a76d-e45d4bacf264 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:20.332225 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e566e-1a84-4f59-a76d-e45d4bacf264 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:20.332624 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682e566e-1a84-4f59-a76d-e45d4bacf264 admin admin] PATH_INFO: `/v3/projects/9c84e3e932ad4cbeb1a750d2ed2615b8` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:20.333707 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e566e-1a84-4f59-a76d-e45d4bacf264 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c84e3e932ad4cbeb1a750d2ed2615b8)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:20.335651 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682e566e-1a84-4f59-a76d-e45d4bacf264 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:20.386452 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-682e566e-1a84-4f59-a76d-e45d4bacf264 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c84e3e932ad4cbeb1a750d2ed2615b8', 'request_id': 'req-682e566e-1a84-4f59-a76d-e45d4bacf264'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:20.392590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-682e566e-1a84-4f59-a76d-e45d4bacf264 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c84e3e932ad4cbeb1a750d2ed2615b8', 'request_id': 'req-682e566e-1a84-4f59-a76d-e45d4bacf264'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:20.401213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-682e566e-1a84-4f59-a76d-e45d4bacf264 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c84e3e932ad4cbeb1a750d2ed2615b8', 'request_id': 'req-682e566e-1a84-4f59-a76d-e45d4bacf264'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:20.416085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-682e566e-1a84-4f59-a76d-e45d4bacf264 admin admin] The token cache is being invalidate because project 9c84e3e932ad4cbeb1a750d2ed2615b8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:08:20.416374 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-682e566e-1a84-4f59-a76d-e45d4bacf264 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:20.417032 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-682e566e-1a84-4f59-a76d-e45d4bacf264 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:20.418628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1020/2046] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:08:20 2022] DELETE /identity/v3/projects/9c84e3e932ad4cbeb1a750d2ed2615b8 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:20.423859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323732e-15f6-4ff3-a3af-3fc2eb0806b9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:20.424156 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323732e-15f6-4ff3-a3af-3fc2eb0806b9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:20.424459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3323732e-15f6-4ff3-a3af-3fc2eb0806b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:20.444263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3323732e-15f6-4ff3-a3af-3fc2eb0806b9 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:20.445391 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3323732e-15f6-4ff3-a3af-3fc2eb0806b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:20.478787 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:20.505558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1027/2047] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:20.513320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbe74c-7667-4c40-9d53-a1b9944510a0 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:20.513610 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbe74c-7667-4c40-9d53-a1b9944510a0 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:20.513889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccdbe74c-7667-4c40-9d53-a1b9944510a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:20.533115 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ccdbe74c-7667-4c40-9d53-a1b9944510a0 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:20.534328 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ccdbe74c-7667-4c40-9d53-a1b9944510a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:20.542352 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1021/2048] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:20.546499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323732e-15f6-4ff3-a3af-3fc2eb0806b9 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:20.556345 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3323732e-15f6-4ff3-a3af-3fc2eb0806b9 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:20.561358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04b41c-89d3-4c8f-bf06-30e535bdc0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:20.562016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04b41c-89d3-4c8f-bf06-30e535bdc0d6 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:20.562316 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04b41c-89d3-4c8f-bf06-30e535bdc0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:20.562607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df04b41c-89d3-4c8f-bf06-30e535bdc0d6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:20.563345 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04b41c-89d3-4c8f-bf06-30e535bdc0d6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:08:20.563644 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04b41c-89d3-4c8f-bf06-30e535bdc0d6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:20.565241 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df04b41c-89d3-4c8f-bf06-30e535bdc0d6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:20.576969 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1028/2049] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:08:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:08:20.581725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdbe74c-7667-4c40-9d53-a1b9944510a0 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:20.591178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccdbe74c-7667-4c40-9d53-a1b9944510a0 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:20.596789 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1be378-5ba7-4fda-a5ae-58b9c79faca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:20.597457 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1be378-5ba7-4fda-a5ae-58b9c79faca9 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:20.597742 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1be378-5ba7-4fda-a5ae-58b9c79faca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:20.598017 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1be378-5ba7-4fda-a5ae-58b9c79faca9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:20.598782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1be378-5ba7-4fda-a5ae-58b9c79faca9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:20.600676 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1be378-5ba7-4fda-a5ae-58b9c79faca9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:20.630986 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1022/2050] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:08:20 2022] POST /identity/v3/projects => generated 371 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:08:20.636075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04b41c-89d3-4c8f-bf06-30e535bdc0d6 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:20.647025 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df04b41c-89d3-4c8f-bf06-30e535bdc0d6 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:20.706042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e4893-98f9-4a30-a564-d9a6e47fcbe5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:20.706657 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e4893-98f9-4a30-a564-d9a6e47fcbe5 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:20.706948 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e4893-98f9-4a30-a564-d9a6e47fcbe5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:20.707225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9e4893-98f9-4a30-a564-d9a6e47fcbe5 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:20.707936 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e4893-98f9-4a30-a564-d9a6e47fcbe5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:20.709851 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9e4893-98f9-4a30-a564-d9a6e47fcbe5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:20.728579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1029/2051] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:08:20 2022] POST /identity/v3/users => generated 286 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:08:20.732921 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1be378-5ba7-4fda-a5ae-58b9c79faca9 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:20.743527 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1be378-5ba7-4fda-a5ae-58b9c79faca9 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:20.748355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f9a8b-a072-49ea-a8fa-403c65780832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:20.749021 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f9a8b-a072-49ea-a8fa-403c65780832 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:20.749304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f9a8b-a072-49ea-a8fa-403c65780832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:20.749586 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e6f9a8b-a072-49ea-a8fa-403c65780832 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:20.750332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f9a8b-a072-49ea-a8fa-403c65780832 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:20.752496 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e6f9a8b-a072-49ea-a8fa-403c65780832 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:20.762945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1023/2052] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:08:20.767543 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e4893-98f9-4a30-a564-d9a6e47fcbe5 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:20.777655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9e4893-98f9-4a30-a564-d9a6e47fcbe5 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:20.782648 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3899b6c-1abc-46dd-90fd-bdd94b19a547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:20.783401 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3899b6c-1abc-46dd-90fd-bdd94b19a547 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:20.783691 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3899b6c-1abc-46dd-90fd-bdd94b19a547 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:20.783972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3899b6c-1abc-46dd-90fd-bdd94b19a547 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:20.784721 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3899b6c-1abc-46dd-90fd-bdd94b19a547 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:20.786326 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3899b6c-1abc-46dd-90fd-bdd94b19a547 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:20.796469 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1030/2053] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:08:20 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:08:20.800629 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f9a8b-a072-49ea-a8fa-403c65780832 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:20.810921 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e6f9a8b-a072-49ea-a8fa-403c65780832 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:20.816732 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d511b-bab6-4792-acb7-46e8b6b20c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:20.817650 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d511b-bab6-4792-acb7-46e8b6b20c60 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:20.817937 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d511b-bab6-4792-acb7-46e8b6b20c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:20.818224 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea6d511b-bab6-4792-acb7-46e8b6b20c60 admin admin] PATH_INFO: `/v3/projects/2c01634e004b43239e6912ee0a9d1ffd/users/e12ac3751780495081b2ef41d13ba360/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:20.820861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:20.840630 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6d511b-bab6-4792-acb7-46e8b6b20c60 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2c01634e004b43239e6912ee0a9d1ffd, user_id=e12ac3751780495081b2ef41d13ba360, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:20.843369 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea6d511b-bab6-4792-acb7-46e8b6b20c60 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:20.856416 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1024/2054] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:08:20 2022] PUT /identity/v3/projects/2c01634e004b43239e6912ee0a9d1ffd/users/e12ac3751780495081b2ef41d13ba360/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:20.861771 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3bad4-cbd0-4450-911c-f2154303dfb9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:20.862086 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3bad4-cbd0-4450-911c-f2154303dfb9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:20.862373 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a3bad4-cbd0-4450-911c-f2154303dfb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:20.880846 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-23a3bad4-cbd0-4450-911c-f2154303dfb9 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:20.882086 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-23a3bad4-cbd0-4450-911c-f2154303dfb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:20.944984 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1031/2055] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:20.955775 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d511b-bab6-4792-acb7-46e8b6b20c60 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:20.957753 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ea6d511b-bab6-4792-acb7-46e8b6b20c60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:21.050938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea6d511b-bab6-4792-acb7-46e8b6b20c60 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:21.055135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41381b57-ebdd-49b4-a42b-632ca3c6e1fd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:21.055817 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41381b57-ebdd-49b4-a42b-632ca3c6e1fd service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:21.056130 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41381b57-ebdd-49b4-a42b-632ca3c6e1fd service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:21.056410 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41381b57-ebdd-49b4-a42b-632ca3c6e1fd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:21.067629 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41381b57-ebdd-49b4-a42b-632ca3c6e1fd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:21.069264 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41381b57-ebdd-49b4-a42b-632ca3c6e1fd service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:21.083678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1025/2056] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:08:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:08:25.730763 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477ea56-b7f5-4965-af5e-38b7b291bb43 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:25.731336 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477ea56-b7f5-4965-af5e-38b7b291bb43 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:25.731440 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2477ea56-b7f5-4965-af5e-38b7b291bb43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:25.763095 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-2477ea56-b7f5-4965-af5e-38b7b291bb43 None None] MFA Rules not processed for user `e12ac3751780495081b2ef41d13ba360`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:25.764434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2477ea56-b7f5-4965-af5e-38b7b291bb43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:25.803575 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:25.837884 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1032/2057] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:25 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:25.842215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41381b57-ebdd-49b4-a42b-632ca3c6e1fd service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:25.853168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41381b57-ebdd-49b4-a42b-632ca3c6e1fd service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:25.858112 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea2803-bc64-49d6-8705-5c87f52127cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:25.858919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea2803-bc64-49d6-8705-5c87f52127cc admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:25.859277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea2803-bc64-49d6-8705-5c87f52127cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:25.859581 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ea2803-bc64-49d6-8705-5c87f52127cc admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:25.860419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea2803-bc64-49d6-8705-5c87f52127cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:25.862868 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3ea2803-bc64-49d6-8705-5c87f52127cc admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:25.905726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1026/2058] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:08:25 2022] POST /identity/v3/projects => generated 371 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:08:25.907831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477ea56-b7f5-4965-af5e-38b7b291bb43 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:25.927360 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2477ea56-b7f5-4965-af5e-38b7b291bb43 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:26.010163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a466f80-daf0-48b8-b710-86293024d95c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:26.010906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a466f80-daf0-48b8-b710-86293024d95c admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:26.011228 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a466f80-daf0-48b8-b710-86293024d95c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:26.011521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a466f80-daf0-48b8-b710-86293024d95c admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:26.012233 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a466f80-daf0-48b8-b710-86293024d95c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:26.014217 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a466f80-daf0-48b8-b710-86293024d95c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:26.042125 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1033/2059] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:08:25 2022] POST /identity/v3/users => generated 292 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:08:26.042920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea2803-bc64-49d6-8705-5c87f52127cc admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:26.056297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ea2803-bc64-49d6-8705-5c87f52127cc admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:26.062261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7819f6-fd6e-4f71-9466-e9d61853b760 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:26.063219 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7819f6-fd6e-4f71-9466-e9d61853b760 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:26.063682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7819f6-fd6e-4f71-9466-e9d61853b760 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:26.064136 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad7819f6-fd6e-4f71-9466-e9d61853b760 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:26.065891 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7819f6-fd6e-4f71-9466-e9d61853b760 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:26.067714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad7819f6-fd6e-4f71-9466-e9d61853b760 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:26.087869 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1027/2060] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:08:26.092726 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a466f80-daf0-48b8-b710-86293024d95c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:26.109195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a466f80-daf0-48b8-b710-86293024d95c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:26.115032 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ffb7d6-9ceb-4421-9da4-79fffc99bf10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:26.115944 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffb7d6-9ceb-4421-9da4-79fffc99bf10 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:26.116230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffb7d6-9ceb-4421-9da4-79fffc99bf10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:26.116512 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ffb7d6-9ceb-4421-9da4-79fffc99bf10 admin admin] PATH_INFO: `/v3/projects/b08b03e04f394198bdf9a320a3a4064b/users/24fc8e511f2d42fbbf66973fea1404f4/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:26.137576 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ffb7d6-9ceb-4421-9da4-79fffc99bf10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b08b03e04f394198bdf9a320a3a4064b, user_id=24fc8e511f2d42fbbf66973fea1404f4, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:26.140610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ffb7d6-9ceb-4421-9da4-79fffc99bf10 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:26.162048 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1034/2061] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:08:26 2022] PUT /identity/v3/projects/b08b03e04f394198bdf9a320a3a4064b/users/24fc8e511f2d42fbbf66973fea1404f4/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:08:26.166104 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7819f6-fd6e-4f71-9466-e9d61853b760 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:26.177725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad7819f6-fd6e-4f71-9466-e9d61853b760 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:26.240514 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aaa9f5f-0048-4a28-98fe-7f3d0fc6cb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:26.241116 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa9f5f-0048-4a28-98fe-7f3d0fc6cb3b admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:26.241466 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa9f5f-0048-4a28-98fe-7f3d0fc6cb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:26.241792 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aaa9f5f-0048-4a28-98fe-7f3d0fc6cb3b admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:26.242807 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaa9f5f-0048-4a28-98fe-7f3d0fc6cb3b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:26.244693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aaa9f5f-0048-4a28-98fe-7f3d0fc6cb3b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:26.258281 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1028/2062] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:08:26 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:08:26.262505 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ffb7d6-9ceb-4421-9da4-79fffc99bf10 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:26.274060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ffb7d6-9ceb-4421-9da4-79fffc99bf10 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:26.279646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fcd7d-9b55-4ba3-9b2f-954b159cb668 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:26.280560 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fcd7d-9b55-4ba3-9b2f-954b159cb668 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:26.280852 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fcd7d-9b55-4ba3-9b2f-954b159cb668 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:26.281187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f20fcd7d-9b55-4ba3-9b2f-954b159cb668 admin admin] PATH_INFO: `/v3/projects/b08b03e04f394198bdf9a320a3a4064b/users/24fc8e511f2d42fbbf66973fea1404f4/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:26.283960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20fcd7d-9b55-4ba3-9b2f-954b159cb668 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b08b03e04f394198bdf9a320a3a4064b, user_id=24fc8e511f2d42fbbf66973fea1404f4, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:26.286928 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f20fcd7d-9b55-4ba3-9b2f-954b159cb668 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:26.298000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1035/2063] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:08:26 2022] PUT /identity/v3/projects/b08b03e04f394198bdf9a320a3a4064b/users/24fc8e511f2d42fbbf66973fea1404f4/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:28.669590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf966966-2d14-4256-af16-4c55203acde1 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:28.670098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf966966-2d14-4256-af16-4c55203acde1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:28.671253 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf966966-2d14-4256-af16-4c55203acde1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:28.702803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-bf966966-2d14-4256-af16-4c55203acde1 None None] MFA Rules not processed for user `52076f59f68f4d77a0dadbf7a98d2b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:28.704739 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bf966966-2d14-4256-af16-4c55203acde1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:28.764244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:28.789210 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1029/2064] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 11 14:08:28 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:28.838673 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fcd7d-9b55-4ba3-9b2f-954b159cb668 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:28.841983 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f20fcd7d-9b55-4ba3-9b2f-954b159cb668 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:28.960064 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f20fcd7d-9b55-4ba3-9b2f-954b159cb668 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:28.975556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0608a2-6a76-4c79-844b-0fdb2bdc2252 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c1369033a44c1c81d4c1c9ff2bbedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:28.976203 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0608a2-6a76-4c79-844b-0fdb2bdc2252 service swift] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:28.976486 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0608a2-6a76-4c79-844b-0fdb2bdc2252 service swift] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:28.976766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0608a2-6a76-4c79-844b-0fdb2bdc2252 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:28.993248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0608a2-6a76-4c79-844b-0fdb2bdc2252 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:28.995798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a0608a2-6a76-4c79-844b-0fdb2bdc2252 service swift] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:29.011376 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1036/2065] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 11 14:08:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 14:08:30.292565 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89279144-a171-4385-bd97-75a4dcd379fa None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:30.293482 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89279144-a171-4385-bd97-75a4dcd379fa None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:30.293924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89279144-a171-4385-bd97-75a4dcd379fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:30.331073 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-89279144-a171-4385-bd97-75a4dcd379fa None None] MFA Rules not processed for user `52076f59f68f4d77a0dadbf7a98d2b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:30.333174 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-89279144-a171-4385-bd97-75a4dcd379fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:30.370940 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1030/2066] 10.222.0.49 () {58 vars in 967 bytes} [Mon Jul 11 14:08:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:08:30.403778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0608a2-6a76-4c79-844b-0fdb2bdc2252 service swift] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:30.425726 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0608a2-6a76-4c79-844b-0fdb2bdc2252 service swift] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:30.434294 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a5144-1644-4ede-8402-343fbf43f48f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c1369033a44c1c81d4c1c9ff2bbedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:30.435175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a5144-1644-4ede-8402-343fbf43f48f service swift] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:30.435465 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a5144-1644-4ede-8402-343fbf43f48f service swift] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:30.435748 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-385a5144-1644-4ede-8402-343fbf43f48f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:30.445857 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a5144-1644-4ede-8402-343fbf43f48f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:30.455066 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-385a5144-1644-4ede-8402-343fbf43f48f service swift] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:30.478944 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1037/2067] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 11 14:08:30 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 76 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 14:08:30.941582 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21da5a4-cf42-4c8a-b043-db9fe438f1d1 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:30.941918 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21da5a4-cf42-4c8a-b043-db9fe438f1d1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:30.942395 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21da5a4-cf42-4c8a-b043-db9fe438f1d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:30.992435 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a21da5a4-cf42-4c8a-b043-db9fe438f1d1 None None] MFA Rules not processed for user `52076f59f68f4d77a0dadbf7a98d2b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:30.996924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a21da5a4-cf42-4c8a-b043-db9fe438f1d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:31.018337 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1031/2068] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 11 14:08:30 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:31.038704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a5144-1644-4ede-8402-343fbf43f48f service swift] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:31.048810 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:31.058627 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-385a5144-1644-4ede-8402-343fbf43f48f service swift] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:31.072697 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a76d8-dc10-4252-8055-184f5dbba318 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c1369033a44c1c81d4c1c9ff2bbedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:31.073707 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a76d8-dc10-4252-8055-184f5dbba318 service swift] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:31.074968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a76d8-dc10-4252-8055-184f5dbba318 service swift] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:31.077424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f60a76d8-dc10-4252-8055-184f5dbba318 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:31.106490 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a76d8-dc10-4252-8055-184f5dbba318 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:31.111456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f60a76d8-dc10-4252-8055-184f5dbba318 service swift] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:31.137228 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1038/2069] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 11 14:08:31 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 14:08:32.054774 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f298690-42fd-4d20-b0f3-e4273fe803fc None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:32.055719 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f298690-42fd-4d20-b0f3-e4273fe803fc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:32.056301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f298690-42fd-4d20-b0f3-e4273fe803fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:32.099231 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4f298690-42fd-4d20-b0f3-e4273fe803fc None None] MFA Rules not processed for user `52076f59f68f4d77a0dadbf7a98d2b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:32.100865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4f298690-42fd-4d20-b0f3-e4273fe803fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:32.110556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1032/2070] 10.222.0.49 () {58 vars in 967 bytes} [Mon Jul 11 14:08:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:32.122804 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a76d8-dc10-4252-8055-184f5dbba318 service swift] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:32.142348 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f60a76d8-dc10-4252-8055-184f5dbba318 service swift] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:32.157096 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5220e7-24fb-4b1b-8892-5db475a2655f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c1369033a44c1c81d4c1c9ff2bbedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:32.160204 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5220e7-24fb-4b1b-8892-5db475a2655f service swift] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:32.166765 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5220e7-24fb-4b1b-8892-5db475a2655f service swift] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:32.167237 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5220e7-24fb-4b1b-8892-5db475a2655f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:32.178993 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5220e7-24fb-4b1b-8892-5db475a2655f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:32.183749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5220e7-24fb-4b1b-8892-5db475a2655f service swift] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:32.196870 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1039/2071] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 11 14:08:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 75 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 14:08:33.058524 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228c441-ce57-4cea-85ea-31765acd87bf None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:33.059092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228c441-ce57-4cea-85ea-31765acd87bf None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:33.059355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0228c441-ce57-4cea-85ea-31765acd87bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:33.108441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0228c441-ce57-4cea-85ea-31765acd87bf None None] MFA Rules not processed for user `24fc8e511f2d42fbbf66973fea1404f4`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:33.109857 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0228c441-ce57-4cea-85ea-31765acd87bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:33.264825 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1033/2072] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:33 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:33.278614 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5220e7-24fb-4b1b-8892-5db475a2655f service swift] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:33.280641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dd5220e7-24fb-4b1b-8892-5db475a2655f service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:33.413606 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5220e7-24fb-4b1b-8892-5db475a2655f service swift] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:33.422029 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3822ecaa-2e25-4da0-89cb-39b9c3878033 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:33.423134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822ecaa-2e25-4da0-89cb-39b9c3878033 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:33.423534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822ecaa-2e25-4da0-89cb-39b9c3878033 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:33.423895 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3822ecaa-2e25-4da0-89cb-39b9c3878033 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:33.566035 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822ecaa-2e25-4da0-89cb-39b9c3878033 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:33.567998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3822ecaa-2e25-4da0-89cb-39b9c3878033 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:33.582214 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1040/2073] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:08:33 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:08:35.030623 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228c441-ce57-4cea-85ea-31765acd87bf None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:35.032865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0228c441-ce57-4cea-85ea-31765acd87bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:35.204737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0228c441-ce57-4cea-85ea-31765acd87bf None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:35.214164 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b609360-e1e6-42d7-a986-f48b401684a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:35.215584 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b609360-e1e6-42d7-a986-f48b401684a6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:35.216092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b609360-e1e6-42d7-a986-f48b401684a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:35.216666 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b609360-e1e6-42d7-a986-f48b401684a6 admin admin] PATH_INFO: `/v3/users/e1fcaa2414f44632b0ed99f7f150a013` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:35.220388 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b609360-e1e6-42d7-a986-f48b401684a6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e1fcaa2414f44632b0ed99f7f150a013)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:35.222733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b609360-e1e6-42d7-a986-f48b401684a6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:35.235645 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25942f-b5e1-45df-a243-8434aadb4ef9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:35.236132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25942f-b5e1-45df-a243-8434aadb4ef9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:35.237031 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f25942f-b5e1-45df-a243-8434aadb4ef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:35.276651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4f25942f-b5e1-45df-a243-8434aadb4ef9 None None] MFA Rules not processed for user `e12ac3751780495081b2ef41d13ba360`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:35.279409 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4f25942f-b5e1-45df-a243-8434aadb4ef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:35.295965 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1041/2074] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:35 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 62 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:08:35.343314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0b609360-e1e6-42d7-a986-f48b401684a6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e1fcaa2414f44632b0ed99f7f150a013'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:35.346170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0b609360-e1e6-42d7-a986-f48b401684a6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e1fcaa2414f44632b0ed99f7f150a013'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:35.348998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02facc-4e8b-4dda-92b4-2453b6a4d092 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:35.348998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02facc-4e8b-4dda-92b4-2453b6a4d092 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:35.349335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d02facc-4e8b-4dda-92b4-2453b6a4d092 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:35.351219 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1042/2075] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 11 14:08:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:08:35.361781 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0b609360-e1e6-42d7-a986-f48b401684a6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e1fcaa2414f44632b0ed99f7f150a013'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:35.362643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0b609360-e1e6-42d7-a986-f48b401684a6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e1fcaa2414f44632b0ed99f7f150a013'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:35.369080 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4effe-fd9e-4efe-a57d-4f6a664d5327 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:35.369574 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4effe-fd9e-4efe-a57d-4f6a664d5327 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:35.370609 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:35.370761 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0f4effe-fd9e-4efe-a57d-4f6a664d5327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:35.389275 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0b609360-e1e6-42d7-a986-f48b401684a6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e1fcaa2414f44632b0ed99f7f150a013'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:35.406212 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1034/2076] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:08:35 2022] DELETE /identity/v3/users/e1fcaa2414f44632b0ed99f7f150a013 => generated 0 bytes in 378 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:35.410961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d6890-2591-419e-bb83-bc319abd58b4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:35.411362 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d6890-2591-419e-bb83-bc319abd58b4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:35.411763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a3d6890-2591-419e-bb83-bc319abd58b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:35.428064 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c0f4effe-fd9e-4efe-a57d-4f6a664d5327 None None] MFA Rules not processed for user `ee29b7f1cea64d6781e1df5c8021363f`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:35.430645 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c0f4effe-fd9e-4efe-a57d-4f6a664d5327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:35.445570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3a3d6890-2591-419e-bb83-bc319abd58b4 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:35.447516 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3a3d6890-2591-419e-bb83-bc319abd58b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:35.534612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1035/2077] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:08:35.535652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d6890-2591-419e-bb83-bc319abd58b4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:35.536994 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3a3d6890-2591-419e-bb83-bc319abd58b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:35.542044 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1043/2078] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 11 14:08:35 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:35.543100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4effe-fd9e-4efe-a57d-4f6a664d5327 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:35.545160 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c0f4effe-fd9e-4efe-a57d-4f6a664d5327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:35.552133 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a3d6890-2591-419e-bb83-bc319abd58b4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:35.557702 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f193fde-ba3d-46af-b0d4-240d58025087 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:35.558715 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f193fde-ba3d-46af-b0d4-240d58025087 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:35.559069 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f193fde-ba3d-46af-b0d4-240d58025087 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:35.559422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f193fde-ba3d-46af-b0d4-240d58025087 admin admin] PATH_INFO: `/v3/users/170f249dccd4498990a8344a18c73059` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:35.560796 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f193fde-ba3d-46af-b0d4-240d58025087 admin admin] RBAC: Authorizing `identity:delete_user(user_id=170f249dccd4498990a8344a18c73059)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:35.562718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:35.562864 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f193fde-ba3d-46af-b0d4-240d58025087 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:35.571124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0f4effe-fd9e-4efe-a57d-4f6a664d5327 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:35.578624 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3532d9-e934-4513-b58b-561b6b81e6a9 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee29b7f1cea64d6781e1df5c8021363f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:35.580105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3532d9-e934-4513-b58b-561b6b81e6a9 service placement] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:35.580371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3532d9-e934-4513-b58b-561b6b81e6a9 service placement] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:35.580665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3532d9-e934-4513-b58b-561b6b81e6a9 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:35.590252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3532d9-e934-4513-b58b-561b6b81e6a9 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:35.591881 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb3532d9-e934-4513-b58b-561b6b81e6a9 service placement] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:35.604071 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1044/2079] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:08:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:08:35.604955 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3532d9-e934-4513-b58b-561b6b81e6a9 service placement] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:35.606442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bb3532d9-e934-4513-b58b-561b6b81e6a9 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:35.634214 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3f193fde-ba3d-46af-b0d4-240d58025087 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '170f249dccd4498990a8344a18c73059'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:35.635193 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3f193fde-ba3d-46af-b0d4-240d58025087 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '170f249dccd4498990a8344a18c73059'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:35.647026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3f193fde-ba3d-46af-b0d4-240d58025087 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '170f249dccd4498990a8344a18c73059'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:35.647976 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3f193fde-ba3d-46af-b0d4-240d58025087 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '170f249dccd4498990a8344a18c73059'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:35.663862 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3f193fde-ba3d-46af-b0d4-240d58025087 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '170f249dccd4498990a8344a18c73059'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:35.682614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1036/2080] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:08:35 2022] DELETE /identity/v3/users/170f249dccd4498990a8344a18c73059 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:35.694968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb3532d9-e934-4513-b58b-561b6b81e6a9 service placement] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:35.769880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378db8f1-4016-4e07-9dac-c74a0d91f948 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:35.770727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378db8f1-4016-4e07-9dac-c74a0d91f948 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:35.771179 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378db8f1-4016-4e07-9dac-c74a0d91f948 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:35.771567 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378db8f1-4016-4e07-9dac-c74a0d91f948 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:35.773733 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-378db8f1-4016-4e07-9dac-c74a0d91f948 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:35.878122 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378db8f1-4016-4e07-9dac-c74a0d91f948 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:35.880225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378db8f1-4016-4e07-9dac-c74a0d91f948 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:35.900473 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f193fde-ba3d-46af-b0d4-240d58025087 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:35.906139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3f193fde-ba3d-46af-b0d4-240d58025087 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:35.931338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1045/2081] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:08:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:08:35.935559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f193fde-ba3d-46af-b0d4-240d58025087 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:35.956926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100a121-9222-4c2a-99bc-db0bb928517f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:35.957832 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100a121-9222-4c2a-99bc-db0bb928517f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:35.958124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100a121-9222-4c2a-99bc-db0bb928517f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:35.958446 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3100a121-9222-4c2a-99bc-db0bb928517f admin admin] PATH_INFO: `/v3/projects/7136b6633c6e4d399a6d359d750125f0` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:35.962965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3100a121-9222-4c2a-99bc-db0bb928517f admin admin] RBAC: Authorizing `identity:delete_project(project_id=7136b6633c6e4d399a6d359d750125f0)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:35.965877 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3100a121-9222-4c2a-99bc-db0bb928517f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:36.062866 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:36.066676 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3100a121-9222-4c2a-99bc-db0bb928517f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7136b6633c6e4d399a6d359d750125f0', 'request_id': 'req-3100a121-9222-4c2a-99bc-db0bb928517f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:36.078245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3100a121-9222-4c2a-99bc-db0bb928517f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7136b6633c6e4d399a6d359d750125f0', 'request_id': 'req-3100a121-9222-4c2a-99bc-db0bb928517f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:36.091921 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3100a121-9222-4c2a-99bc-db0bb928517f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7136b6633c6e4d399a6d359d750125f0', 'request_id': 'req-3100a121-9222-4c2a-99bc-db0bb928517f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:36.108880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3100a121-9222-4c2a-99bc-db0bb928517f admin admin] The token cache is being invalidate because project 7136b6633c6e4d399a6d359d750125f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:08:36.109184 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3100a121-9222-4c2a-99bc-db0bb928517f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:36.109874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3100a121-9222-4c2a-99bc-db0bb928517f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:36.112548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1037/2082] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:08:35 2022] DELETE /identity/v3/projects/7136b6633c6e4d399a6d359d750125f0 => generated 0 bytes in 212 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:36.117320 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083268f-1a9b-49d1-8779-45f393b4bd63 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:36.117820 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083268f-1a9b-49d1-8779-45f393b4bd63 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:36.118253 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8083268f-1a9b-49d1-8779-45f393b4bd63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:36.140300 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8083268f-1a9b-49d1-8779-45f393b4bd63 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:36.142590 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8083268f-1a9b-49d1-8779-45f393b4bd63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:36.212854 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:36.224532 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100a121-9222-4c2a-99bc-db0bb928517f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:36.230248 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3100a121-9222-4c2a-99bc-db0bb928517f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:36.238021 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1046/2083] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:36.259976 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775c204-70c8-442c-8205-80663a17fce0 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:36.263593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775c204-70c8-442c-8205-80663a17fce0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:36.265945 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3775c204-70c8-442c-8205-80663a17fce0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:36.283009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3100a121-9222-4c2a-99bc-db0bb928517f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:36.303204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3689b2-b901-40a1-9656-391a6499579e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:36.304107 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3689b2-b901-40a1-9656-391a6499579e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:36.304399 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3689b2-b901-40a1-9656-391a6499579e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:36.304678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f3689b2-b901-40a1-9656-391a6499579e admin admin] PATH_INFO: `/v3/projects/809cb51806b54475bc876aaa27c556c0` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:36.306205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3775c204-70c8-442c-8205-80663a17fce0 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:36.307682 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3775c204-70c8-442c-8205-80663a17fce0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:36.312678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3689b2-b901-40a1-9656-391a6499579e admin admin] RBAC: Authorizing `identity:delete_project(project_id=809cb51806b54475bc876aaa27c556c0)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:36.318037 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f3689b2-b901-40a1-9656-391a6499579e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:36.333368 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1047/2084] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:36.339154 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775c204-70c8-442c-8205-80663a17fce0 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:36.357017 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3775c204-70c8-442c-8205-80663a17fce0 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:36.373788 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab429a-9ac9-4007-83cd-5dfc5283a697 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:36.374924 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab429a-9ac9-4007-83cd-5dfc5283a697 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:36.375379 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab429a-9ac9-4007-83cd-5dfc5283a697 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:36.375876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5aab429a-9ac9-4007-83cd-5dfc5283a697 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:36.376962 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab429a-9ac9-4007-83cd-5dfc5283a697 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:08:36.377476 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab429a-9ac9-4007-83cd-5dfc5283a697 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:36.380523 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5aab429a-9ac9-4007-83cd-5dfc5283a697 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:36.407623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1048/2085] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:08:36 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:08:36.408886 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab429a-9ac9-4007-83cd-5dfc5283a697 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:36.421789 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5aab429a-9ac9-4007-83cd-5dfc5283a697 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:36.428317 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1f3689b2-b901-40a1-9656-391a6499579e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '809cb51806b54475bc876aaa27c556c0', 'request_id': 'req-1f3689b2-b901-40a1-9656-391a6499579e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:36.442693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1f3689b2-b901-40a1-9656-391a6499579e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '809cb51806b54475bc876aaa27c556c0', 'request_id': 'req-1f3689b2-b901-40a1-9656-391a6499579e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:36.452511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1f3689b2-b901-40a1-9656-391a6499579e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '809cb51806b54475bc876aaa27c556c0', 'request_id': 'req-1f3689b2-b901-40a1-9656-391a6499579e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:36.489175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1f3689b2-b901-40a1-9656-391a6499579e admin admin] The token cache is being invalidate because project 809cb51806b54475bc876aaa27c556c0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:08:36.489475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1f3689b2-b901-40a1-9656-391a6499579e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:36.491443 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1f3689b2-b901-40a1-9656-391a6499579e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:36.495079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1038/2086] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:08:36 2022] DELETE /identity/v3/projects/809cb51806b54475bc876aaa27c556c0 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:36.572285 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e39419-f1e9-41d5-b8cf-e1d06d4c008a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:36.573511 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e39419-f1e9-41d5-b8cf-e1d06d4c008a admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:36.575481 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e39419-f1e9-41d5-b8cf-e1d06d4c008a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:36.576198 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52e39419-f1e9-41d5-b8cf-e1d06d4c008a admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:36.583080 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e39419-f1e9-41d5-b8cf-e1d06d4c008a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:36.588652 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52e39419-f1e9-41d5-b8cf-e1d06d4c008a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:36.661722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1049/2087] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:08:36 2022] POST /identity/v3/projects => generated 365 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:08:36.663376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3689b2-b901-40a1-9656-391a6499579e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:36.665299 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1f3689b2-b901-40a1-9656-391a6499579e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:36.704417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f3689b2-b901-40a1-9656-391a6499579e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:36.867305 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448474d7-15a1-44fd-9e32-62cb4c67412a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:36.868788 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448474d7-15a1-44fd-9e32-62cb4c67412a admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:36.869482 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448474d7-15a1-44fd-9e32-62cb4c67412a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:36.870004 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448474d7-15a1-44fd-9e32-62cb4c67412a admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:36.871673 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448474d7-15a1-44fd-9e32-62cb4c67412a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:36.875793 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448474d7-15a1-44fd-9e32-62cb4c67412a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:36.909116 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1039/2088] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:08:36 2022] POST /identity/v3/users => generated 283 bytes in 247 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:08:36.913176 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e39419-f1e9-41d5-b8cf-e1d06d4c008a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:36.927150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52e39419-f1e9-41d5-b8cf-e1d06d4c008a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:36.954197 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2e8a0-5c73-4a42-956f-4c7540409385 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:36.955139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2e8a0-5c73-4a42-956f-4c7540409385 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:36.955428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2e8a0-5c73-4a42-956f-4c7540409385 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:36.955708 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c2e8a0-5c73-4a42-956f-4c7540409385 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:36.957744 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2e8a0-5c73-4a42-956f-4c7540409385 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:36.960356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c2e8a0-5c73-4a42-956f-4c7540409385 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:36.972781 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1050/2089] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 60 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:08:36.976962 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448474d7-15a1-44fd-9e32-62cb4c67412a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:36.987655 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448474d7-15a1-44fd-9e32-62cb4c67412a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:36.992317 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32711ebd-5ebe-467e-836d-33efa3a63f3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:36.992990 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32711ebd-5ebe-467e-836d-33efa3a63f3c admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:36.993292 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32711ebd-5ebe-467e-836d-33efa3a63f3c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:36.993572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32711ebd-5ebe-467e-836d-33efa3a63f3c admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:36.994482 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32711ebd-5ebe-467e-836d-33efa3a63f3c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:36.999488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32711ebd-5ebe-467e-836d-33efa3a63f3c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:37.011793 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1040/2090] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:08:36 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:08:37.015729 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2e8a0-5c73-4a42-956f-4c7540409385 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:37.030399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c2e8a0-5c73-4a42-956f-4c7540409385 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:37.038586 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6293696-8c8f-49b3-a5e1-e5041ab4d166 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:37.039826 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6293696-8c8f-49b3-a5e1-e5041ab4d166 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:37.040186 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6293696-8c8f-49b3-a5e1-e5041ab4d166 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:37.040535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6293696-8c8f-49b3-a5e1-e5041ab4d166 admin admin] PATH_INFO: `/v3/projects/90b8d008e7a142d881ddbae53f5b3dc5/users/2fe9fe4b9d0049f09d01dde4be16083a/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:37.087620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6293696-8c8f-49b3-a5e1-e5041ab4d166 admin admin] RBAC: Authorizing `identity:create_grant(project_id=90b8d008e7a142d881ddbae53f5b3dc5, user_id=2fe9fe4b9d0049f09d01dde4be16083a, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:37.090376 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6293696-8c8f-49b3-a5e1-e5041ab4d166 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:37.105863 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1051/2091] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:08:37 2022] PUT /identity/v3/projects/90b8d008e7a142d881ddbae53f5b3dc5/users/2fe9fe4b9d0049f09d01dde4be16083a/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:37.116270 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b067c5f-a56a-4b77-a085-e8bde24a7ed4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:37.117859 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b067c5f-a56a-4b77-a085-e8bde24a7ed4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:37.117859 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b067c5f-a56a-4b77-a085-e8bde24a7ed4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:37.153453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-6b067c5f-a56a-4b77-a085-e8bde24a7ed4 None None] MFA Rules not processed for user `2fe9fe4b9d0049f09d01dde4be16083a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:37.154755 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6b067c5f-a56a-4b77-a085-e8bde24a7ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:37.181046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=11,keystone:SELECT=84,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:37.228765 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1041/2092] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:37 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:37.239599 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6293696-8c8f-49b3-a5e1-e5041ab4d166 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:37.241571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a6293696-8c8f-49b3-a5e1-e5041ab4d166 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:37.310951 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6293696-8c8f-49b3-a5e1-e5041ab4d166 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:37.315339 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130486b0-19f4-42e7-8878-d7f19de56d20 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:37.316007 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130486b0-19f4-42e7-8878-d7f19de56d20 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:37.316329 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130486b0-19f4-42e7-8878-d7f19de56d20 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:37.316610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130486b0-19f4-42e7-8878-d7f19de56d20 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:37.326116 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130486b0-19f4-42e7-8878-d7f19de56d20 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:37.327714 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130486b0-19f4-42e7-8878-d7f19de56d20 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:37.346498 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1052/2093] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:08:37 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:08:47.225475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=16 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:47.338391 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=84,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:53.179083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b067c5f-a56a-4b77-a085-e8bde24a7ed4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:53.180567 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6b067c5f-a56a-4b77-a085-e8bde24a7ed4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:53.246045 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b067c5f-a56a-4b77-a085-e8bde24a7ed4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:53.250715 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e195db-241e-4b44-9c04-3ea4e4f18132 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:53.251641 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e195db-241e-4b44-9c04-3ea4e4f18132 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:53.251956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e195db-241e-4b44-9c04-3ea4e4f18132 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:53.252238 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4e195db-241e-4b44-9c04-3ea4e4f18132 admin admin] PATH_INFO: `/v3/users/db920784425247bfa592700df52ef5e2` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:53.254217 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e195db-241e-4b44-9c04-3ea4e4f18132 admin admin] RBAC: Authorizing `identity:delete_user(user_id=db920784425247bfa592700df52ef5e2)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:53.256432 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4e195db-241e-4b44-9c04-3ea4e4f18132 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:53.304548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b4e195db-241e-4b44-9c04-3ea4e4f18132 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'db920784425247bfa592700df52ef5e2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:53.305206 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b4e195db-241e-4b44-9c04-3ea4e4f18132 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'db920784425247bfa592700df52ef5e2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:53.312551 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b4e195db-241e-4b44-9c04-3ea4e4f18132 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'db920784425247bfa592700df52ef5e2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:53.313237 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b4e195db-241e-4b44-9c04-3ea4e4f18132 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'db920784425247bfa592700df52ef5e2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:53.326029 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b4e195db-241e-4b44-9c04-3ea4e4f18132 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'db920784425247bfa592700df52ef5e2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:53.338373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1042/2094] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:08:53 2022] DELETE /identity/v3/users/db920784425247bfa592700df52ef5e2 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:53.343054 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06106c20-d51f-4715-b8cd-25f987db7dc1 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:53.343527 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06106c20-d51f-4715-b8cd-25f987db7dc1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:53.343621 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06106c20-d51f-4715-b8cd-25f987db7dc1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:53.363017 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-06106c20-d51f-4715-b8cd-25f987db7dc1 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:53.364230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-06106c20-d51f-4715-b8cd-25f987db7dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:53.401724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e195db-241e-4b44-9c04-3ea4e4f18132 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:53.403265 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b4e195db-241e-4b44-9c04-3ea4e4f18132 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:53.426881 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1053/2095] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:08:53.436775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06106c20-d51f-4715-b8cd-25f987db7dc1 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:53.438214 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-06106c20-d51f-4715-b8cd-25f987db7dc1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:53.478784 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4e195db-241e-4b44-9c04-3ea4e4f18132 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:53.483598 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a729587-164f-4912-8ff0-86ccefa2ec9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:53.484479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a729587-164f-4912-8ff0-86ccefa2ec9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:53.484767 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a729587-164f-4912-8ff0-86ccefa2ec9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:53.485047 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a729587-164f-4912-8ff0-86ccefa2ec9a admin admin] PATH_INFO: `/v3/users/2fe9fe4b9d0049f09d01dde4be16083a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:53.486268 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a729587-164f-4912-8ff0-86ccefa2ec9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=2fe9fe4b9d0049f09d01dde4be16083a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:53.488785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a729587-164f-4912-8ff0-86ccefa2ec9a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:53.497397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06106c20-d51f-4715-b8cd-25f987db7dc1 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:53.501071 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-386ea6f9-c895-433e-80f3-6a4315c08408 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:53.501731 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ea6f9-c895-433e-80f3-6a4315c08408 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:53.502017 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ea6f9-c895-433e-80f3-6a4315c08408 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:53.502316 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-386ea6f9-c895-433e-80f3-6a4315c08408 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:53.511565 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:53.512291 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ea6f9-c895-433e-80f3-6a4315c08408 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:53.513817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-386ea6f9-c895-433e-80f3-6a4315c08408 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:53.525341 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1054/2096] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:08:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:08:53.544614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7a729587-164f-4912-8ff0-86ccefa2ec9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2fe9fe4b9d0049f09d01dde4be16083a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:53.545291 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7a729587-164f-4912-8ff0-86ccefa2ec9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2fe9fe4b9d0049f09d01dde4be16083a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:53.553443 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7a729587-164f-4912-8ff0-86ccefa2ec9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2fe9fe4b9d0049f09d01dde4be16083a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:53.554145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7a729587-164f-4912-8ff0-86ccefa2ec9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2fe9fe4b9d0049f09d01dde4be16083a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:53.567716 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7a729587-164f-4912-8ff0-86ccefa2ec9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2fe9fe4b9d0049f09d01dde4be16083a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:53.581044 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1043/2097] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:08:53 2022] DELETE /identity/v3/users/2fe9fe4b9d0049f09d01dde4be16083a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:08:53.586226 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecd0ba-000c-42dc-b3b7-04f5720273f1 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:53.586564 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecd0ba-000c-42dc-b3b7-04f5720273f1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:53.586874 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ecd0ba-000c-42dc-b3b7-04f5720273f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:53.609184 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-f0ecd0ba-000c-42dc-b3b7-04f5720273f1 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:53.610891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f0ecd0ba-000c-42dc-b3b7-04f5720273f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:53.674741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1055/2098] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:53.690239 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a729587-164f-4912-8ff0-86ccefa2ec9a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:53.692108 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7a729587-164f-4912-8ff0-86ccefa2ec9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:53.796983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a729587-164f-4912-8ff0-86ccefa2ec9a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:53.801354 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f81ef11b-9694-4821-b36e-3106fb65d126 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:53.802279 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ef11b-9694-4821-b36e-3106fb65d126 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:53.802623 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ef11b-9694-4821-b36e-3106fb65d126 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:53.802959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f81ef11b-9694-4821-b36e-3106fb65d126 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:53.812893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ef11b-9694-4821-b36e-3106fb65d126 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:53.814431 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f81ef11b-9694-4821-b36e-3106fb65d126 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:53.825274 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1044/2099] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:08:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:08:53.850368 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecd0ba-000c-42dc-b3b7-04f5720273f1 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:53.854525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f0ecd0ba-000c-42dc-b3b7-04f5720273f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:53.909515 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ecd0ba-000c-42dc-b3b7-04f5720273f1 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:53.922158 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f59561-82e2-4b34-8ea3-22f32075c7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:53.923082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f59561-82e2-4b34-8ea3-22f32075c7b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:53.923370 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f59561-82e2-4b34-8ea3-22f32075c7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:53.923647 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f59561-82e2-4b34-8ea3-22f32075c7b3 admin admin] PATH_INFO: `/v3/projects/e3cc52e007fe407db072f8c788b984dd` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:53.924900 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f59561-82e2-4b34-8ea3-22f32075c7b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3cc52e007fe407db072f8c788b984dd)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:53.927670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f59561-82e2-4b34-8ea3-22f32075c7b3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:53.992337 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:54.007167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-52f59561-82e2-4b34-8ea3-22f32075c7b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3cc52e007fe407db072f8c788b984dd', 'request_id': 'req-52f59561-82e2-4b34-8ea3-22f32075c7b3'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:54.020639 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-52f59561-82e2-4b34-8ea3-22f32075c7b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3cc52e007fe407db072f8c788b984dd', 'request_id': 'req-52f59561-82e2-4b34-8ea3-22f32075c7b3'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:54.033660 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-52f59561-82e2-4b34-8ea3-22f32075c7b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3cc52e007fe407db072f8c788b984dd', 'request_id': 'req-52f59561-82e2-4b34-8ea3-22f32075c7b3'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:54.065213 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-52f59561-82e2-4b34-8ea3-22f32075c7b3 admin admin] The token cache is being invalidate because project e3cc52e007fe407db072f8c788b984dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:08:54.065511 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-52f59561-82e2-4b34-8ea3-22f32075c7b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:54.066821 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-52f59561-82e2-4b34-8ea3-22f32075c7b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:54.068766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1056/2100] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:08:53 2022] DELETE /identity/v3/projects/e3cc52e007fe407db072f8c788b984dd => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:54.077051 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ad059-f6b0-4b00-b8d0-e893c01742f9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.078604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ad059-f6b0-4b00-b8d0-e893c01742f9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.078604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ad059-f6b0-4b00-b8d0-e893c01742f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.103195 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-283ad059-f6b0-4b00-b8d0-e893c01742f9 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:54.104513 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-283ad059-f6b0-4b00-b8d0-e893c01742f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:54.204182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1045/2101] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:08:54.212764 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff50e6-1a6c-4acb-9e2f-39ce563e9373 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.213179 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff50e6-1a6c-4acb-9e2f-39ce563e9373 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.213654 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcff50e6-1a6c-4acb-9e2f-39ce563e9373 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.235398 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-dcff50e6-1a6c-4acb-9e2f-39ce563e9373 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:54.236740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dcff50e6-1a6c-4acb-9e2f-39ce563e9373 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:54.246345 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1057/2102] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:54.250444 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ad059-f6b0-4b00-b8d0-e893c01742f9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:54.261915 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ad059-f6b0-4b00-b8d0-e893c01742f9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:54.266913 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb6502-4dad-4cd8-86ac-94cba437602e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:54.267589 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb6502-4dad-4cd8-86ac-94cba437602e admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.267877 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb6502-4dad-4cd8-86ac-94cba437602e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.268154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bb6502-4dad-4cd8-86ac-94cba437602e admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.268921 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb6502-4dad-4cd8-86ac-94cba437602e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:08:54.269229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb6502-4dad-4cd8-86ac-94cba437602e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:54.270892 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bb6502-4dad-4cd8-86ac-94cba437602e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:54.285076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1046/2103] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:08:54.289306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff50e6-1a6c-4acb-9e2f-39ce563e9373 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:54.299092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcff50e6-1a6c-4acb-9e2f-39ce563e9373 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:54.304681 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b9337-2852-418d-843f-59da97236063 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:54.305373 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b9337-2852-418d-843f-59da97236063 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.305661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b9337-2852-418d-843f-59da97236063 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.305933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc8b9337-2852-418d-843f-59da97236063 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.306670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b9337-2852-418d-843f-59da97236063 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:54.309192 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc8b9337-2852-418d-843f-59da97236063 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:54.323376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb6502-4dad-4cd8-86ac-94cba437602e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:54.326565 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b1bb6502-4dad-4cd8-86ac-94cba437602e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:54.345002 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bb6502-4dad-4cd8-86ac-94cba437602e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:54.347553 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1058/2104] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:08:54 2022] POST /identity/v3/projects => generated 385 bytes in 59 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:08:54.351497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b9337-2852-418d-843f-59da97236063 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:54.375272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:54.419195 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb53ffc7-e002-488a-bfae-62cd70d197c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:54.419406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc8b9337-2852-418d-843f-59da97236063 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:54.420089 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53ffc7-e002-488a-bfae-62cd70d197c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.420253 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53ffc7-e002-488a-bfae-62cd70d197c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.420525 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb53ffc7-e002-488a-bfae-62cd70d197c4 admin admin] PATH_INFO: `/v3/projects/90b8d008e7a142d881ddbae53f5b3dc5` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.421692 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53ffc7-e002-488a-bfae-62cd70d197c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=90b8d008e7a142d881ddbae53f5b3dc5)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:54.423650 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb53ffc7-e002-488a-bfae-62cd70d197c4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:54.423964 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96d637-19af-461d-a44e-fc2f6503d2ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:54.424660 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96d637-19af-461d-a44e-fc2f6503d2ff admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.425688 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96d637-19af-461d-a44e-fc2f6503d2ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.425802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f96d637-19af-461d-a44e-fc2f6503d2ff admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.426874 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96d637-19af-461d-a44e-fc2f6503d2ff admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:54.428746 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f96d637-19af-461d-a44e-fc2f6503d2ff admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:54.449228 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1059/2105] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:08:54 2022] POST /identity/v3/users => generated 293 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:08:54.453371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96d637-19af-461d-a44e-fc2f6503d2ff admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:54.464485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f96d637-19af-461d-a44e-fc2f6503d2ff admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:54.475623 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cb53ffc7-e002-488a-bfae-62cd70d197c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '90b8d008e7a142d881ddbae53f5b3dc5', 'request_id': 'req-cb53ffc7-e002-488a-bfae-62cd70d197c4'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:54.481966 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cb53ffc7-e002-488a-bfae-62cd70d197c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '90b8d008e7a142d881ddbae53f5b3dc5', 'request_id': 'req-cb53ffc7-e002-488a-bfae-62cd70d197c4'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:54.490931 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cb53ffc7-e002-488a-bfae-62cd70d197c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '90b8d008e7a142d881ddbae53f5b3dc5', 'request_id': 'req-cb53ffc7-e002-488a-bfae-62cd70d197c4'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:54.504269 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cb53ffc7-e002-488a-bfae-62cd70d197c4 admin admin] The token cache is being invalidate because project 90b8d008e7a142d881ddbae53f5b3dc5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:08:54.504559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cb53ffc7-e002-488a-bfae-62cd70d197c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:54.505310 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cb53ffc7-e002-488a-bfae-62cd70d197c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:08:54.506994 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1047/2106] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:08:54 2022] DELETE /identity/v3/projects/90b8d008e7a142d881ddbae53f5b3dc5 => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:54.512060 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bbb35-d4cc-4302-9c95-d78272b88a27 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.512196 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bbb35-d4cc-4302-9c95-d78272b88a27 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.512390 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc3bbb35-d4cc-4302-9c95-d78272b88a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.527185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269bfff9-54d8-4871-9336-b4f5c6afc574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:54.527868 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bfff9-54d8-4871-9336-b4f5c6afc574 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.528158 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bfff9-54d8-4871-9336-b4f5c6afc574 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.528444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269bfff9-54d8-4871-9336-b4f5c6afc574 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.529185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269bfff9-54d8-4871-9336-b4f5c6afc574 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:54.530865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269bfff9-54d8-4871-9336-b4f5c6afc574 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:54.530992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-fc3bbb35-d4cc-4302-9c95-d78272b88a27 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:54.532225 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fc3bbb35-d4cc-4302-9c95-d78272b88a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:54.541019 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1048/2107] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:54.544402 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1060/2108] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:08:54.547404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bbb35-d4cc-4302-9c95-d78272b88a27 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:54.548778 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fc3bbb35-d4cc-4302-9c95-d78272b88a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:54.548891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab8c74-a14b-4bf2-a412-d05cb983dbc2 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.549109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab8c74-a14b-4bf2-a412-d05cb983dbc2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.549390 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deab8c74-a14b-4bf2-a412-d05cb983dbc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.562235 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc3bbb35-d4cc-4302-9c95-d78272b88a27 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:54.568005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83406dc-fe6f-41ab-83f7-665384fe0a87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:54.568681 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83406dc-fe6f-41ab-83f7-665384fe0a87 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.568968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83406dc-fe6f-41ab-83f7-665384fe0a87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.569250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b83406dc-fe6f-41ab-83f7-665384fe0a87 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.569992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83406dc-fe6f-41ab-83f7-665384fe0a87 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:54.570495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-deab8c74-a14b-4bf2-a412-d05cb983dbc2 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:54.571675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b83406dc-fe6f-41ab-83f7-665384fe0a87 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:54.571906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-deab8c74-a14b-4bf2-a412-d05cb983dbc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:54.585135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1061/2109] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:08:54.585299 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1049/2110] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:08:54.586512 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab8c74-a14b-4bf2-a412-d05cb983dbc2 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:54.591073 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83406dc-fe6f-41ab-83f7-665384fe0a87 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:54.596783 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deab8c74-a14b-4bf2-a412-d05cb983dbc2 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:54.601772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59471fbe-b6da-4ed8-8fd4-93b9a7421fcf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:54.602489 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59471fbe-b6da-4ed8-8fd4-93b9a7421fcf admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.602767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59471fbe-b6da-4ed8-8fd4-93b9a7421fcf admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.603049 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59471fbe-b6da-4ed8-8fd4-93b9a7421fcf admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.603813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59471fbe-b6da-4ed8-8fd4-93b9a7421fcf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:08:54.604128 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59471fbe-b6da-4ed8-8fd4-93b9a7421fcf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:54.604759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b83406dc-fe6f-41ab-83f7-665384fe0a87 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:54.605757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59471fbe-b6da-4ed8-8fd4-93b9a7421fcf admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:54.609499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03105ca2-1e89-4a09-ac23-f2c25bc543c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:54.610414 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03105ca2-1e89-4a09-ac23-f2c25bc543c2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.610699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03105ca2-1e89-4a09-ac23-f2c25bc543c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.610975 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03105ca2-1e89-4a09-ac23-f2c25bc543c2 admin admin] PATH_INFO: `/v3/projects/32353b01d57e434eafdb5f253517ef93/users/365acac78e864078af1a3db4842bc149/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.620033 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1062/2111] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:08:54.624358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59471fbe-b6da-4ed8-8fd4-93b9a7421fcf admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:54.626136 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03105ca2-1e89-4a09-ac23-f2c25bc543c2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=32353b01d57e434eafdb5f253517ef93, user_id=365acac78e864078af1a3db4842bc149, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:54.629535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03105ca2-1e89-4a09-ac23-f2c25bc543c2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:54.631436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=91,keystone:DELETE=4,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:54.635285 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59471fbe-b6da-4ed8-8fd4-93b9a7421fcf admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:54.640995 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87fdf8-ccb8-4c16-aeb2-637650cb6ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:54.641791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87fdf8-ccb8-4c16-aeb2-637650cb6ca6 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.642503 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87fdf8-ccb8-4c16-aeb2-637650cb6ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.642909 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e87fdf8-ccb8-4c16-aeb2-637650cb6ca6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.644399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e87fdf8-ccb8-4c16-aeb2-637650cb6ca6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:54.645033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1050/2112] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:08:54 2022] PUT /identity/v3/projects/32353b01d57e434eafdb5f253517ef93/users/365acac78e864078af1a3db4842bc149/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:54.646922 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e87fdf8-ccb8-4c16-aeb2-637650cb6ca6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:54.650083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9923d67-f83d-4a0d-a85e-2fc8ebe7461e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.650409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9923d67-f83d-4a0d-a85e-2fc8ebe7461e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.650696 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9923d67-f83d-4a0d-a85e-2fc8ebe7461e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.667928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e9923d67-f83d-4a0d-a85e-2fc8ebe7461e None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:54.669170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e9923d67-f83d-4a0d-a85e-2fc8ebe7461e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:54.674620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1063/2113] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:08:54 2022] POST /identity/v3/projects => generated 355 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:08:54.679366 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87fdf8-ccb8-4c16-aeb2-637650cb6ca6 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:54.732234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1051/2114] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:54.742381 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9923d67-f83d-4a0d-a85e-2fc8ebe7461e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:54.743663 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e87fdf8-ccb8-4c16-aeb2-637650cb6ca6 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:54.743778 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e9923d67-f83d-4a0d-a85e-2fc8ebe7461e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:54.748324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf86d0-c058-4f7c-a72b-12a20c5520f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:54.749247 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf86d0-c058-4f7c-a72b-12a20c5520f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.749533 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf86d0-c058-4f7c-a72b-12a20c5520f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.750357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8bf86d0-c058-4f7c-a72b-12a20c5520f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.751072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf86d0-c058-4f7c-a72b-12a20c5520f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:54.753004 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8bf86d0-c058-4f7c-a72b-12a20c5520f7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:54.763970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:54.778766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1064/2115] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:08:54 2022] POST /identity/v3/users => generated 278 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:08:54.780003 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf86d0-c058-4f7c-a72b-12a20c5520f7 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:54.790108 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8bf86d0-c058-4f7c-a72b-12a20c5520f7 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:54.795776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580bb42-8d72-4651-b2c3-8c8e2b10a8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:54.796439 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580bb42-8d72-4651-b2c3-8c8e2b10a8e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.796724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580bb42-8d72-4651-b2c3-8c8e2b10a8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.797053 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4580bb42-8d72-4651-b2c3-8c8e2b10a8e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.797727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580bb42-8d72-4651-b2c3-8c8e2b10a8e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:54.799363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4580bb42-8d72-4651-b2c3-8c8e2b10a8e8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:54.805879 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9923d67-f83d-4a0d-a85e-2fc8ebe7461e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:54.810999 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1065/2116] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:08:54.814945 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580bb42-8d72-4651-b2c3-8c8e2b10a8e8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:54.825821 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4580bb42-8d72-4651-b2c3-8c8e2b10a8e8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:54.831053 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe805d2-3cec-4683-bd1a-1fc97cf1c8a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:54.831758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe805d2-3cec-4683-bd1a-1fc97cf1c8a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.832049 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe805d2-3cec-4683-bd1a-1fc97cf1c8a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.832352 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfe805d2-3cec-4683-bd1a-1fc97cf1c8a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.833101 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe805d2-3cec-4683-bd1a-1fc97cf1c8a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:54.834761 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfe805d2-3cec-4683-bd1a-1fc97cf1c8a9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:54.845710 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1066/2117] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:08:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:08:54.850116 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe805d2-3cec-4683-bd1a-1fc97cf1c8a9 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:54.858902 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7196aa37-bb26-4761-b793-ee673fa1cf2c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:54.859641 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196aa37-bb26-4761-b793-ee673fa1cf2c service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.859923 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196aa37-bb26-4761-b793-ee673fa1cf2c service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.860051 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfe805d2-3cec-4683-bd1a-1fc97cf1c8a9 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:54.860200 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7196aa37-bb26-4761-b793-ee673fa1cf2c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.867155 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff54b1-400a-4e0f-ab0e-13a207b84384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:54.868063 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff54b1-400a-4e0f-ab0e-13a207b84384 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.868356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff54b1-400a-4e0f-ab0e-13a207b84384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.868633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ff54b1-400a-4e0f-ab0e-13a207b84384 admin admin] PATH_INFO: `/v3/projects/a636ba8ac1f344fb84bd8dbe2d86b6d9/users/b53dc890936b4a5d86adb29a5dc688f2/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.879688 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196aa37-bb26-4761-b793-ee673fa1cf2c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:54.881151 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7196aa37-bb26-4761-b793-ee673fa1cf2c service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:54.892666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff54b1-400a-4e0f-ab0e-13a207b84384 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a636ba8ac1f344fb84bd8dbe2d86b6d9, user_id=b53dc890936b4a5d86adb29a5dc688f2, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:54.895114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1052/2118] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:08:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:08:54.895693 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ff54b1-400a-4e0f-ab0e-13a207b84384 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:54.915348 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1067/2119] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:08:54 2022] PUT /identity/v3/projects/a636ba8ac1f344fb84bd8dbe2d86b6d9/users/b53dc890936b4a5d86adb29a5dc688f2/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:08:54.928141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb757b48-f7be-4744-a703-ac23398b4fdd None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:54.928927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb757b48-f7be-4744-a703-ac23398b4fdd None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:54.929280 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb757b48-f7be-4744-a703-ac23398b4fdd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:54.961145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-bb757b48-f7be-4744-a703-ac23398b4fdd None None] MFA Rules not processed for user `b53dc890936b4a5d86adb29a5dc688f2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:54.962517 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bb757b48-f7be-4744-a703-ac23398b4fdd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:55.042708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1053/2120] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:54 2022] POST /identity/v3/auth/tokens => generated 3222 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:55.047629 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff54b1-400a-4e0f-ab0e-13a207b84384 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:55.132855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ff54b1-400a-4e0f-ab0e-13a207b84384 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:55.137969 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db86cb-eb56-42fc-aa08-cab1fb014d0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:55.138757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db86cb-eb56-42fc-aa08-cab1fb014d0b admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:55.139041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db86cb-eb56-42fc-aa08-cab1fb014d0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:55.139308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81db86cb-eb56-42fc-aa08-cab1fb014d0b admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:55.140021 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db86cb-eb56-42fc-aa08-cab1fb014d0b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:55.141951 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81db86cb-eb56-42fc-aa08-cab1fb014d0b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:55.172223 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1068/2121] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:08:55 2022] POST /identity/v3/projects => generated 357 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:08:55.177882 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb757b48-f7be-4744-a703-ac23398b4fdd None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:55.190878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb757b48-f7be-4744-a703-ac23398b4fdd None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:55.262184 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7521dc45-7c5d-4be1-b36a-bd31b5519f17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:55.262932 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521dc45-7c5d-4be1-b36a-bd31b5519f17 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:55.263226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521dc45-7c5d-4be1-b36a-bd31b5519f17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:55.263510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7521dc45-7c5d-4be1-b36a-bd31b5519f17 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:55.264389 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7521dc45-7c5d-4be1-b36a-bd31b5519f17 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:55.266493 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7521dc45-7c5d-4be1-b36a-bd31b5519f17 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:55.279645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:55.296148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1054/2122] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:08:55 2022] POST /identity/v3/users => generated 285 bytes in 119 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:08:55.300323 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db86cb-eb56-42fc-aa08-cab1fb014d0b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:55.313492 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81db86cb-eb56-42fc-aa08-cab1fb014d0b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:55.319784 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e011c2-ef08-4f0b-ad2d-81c4f754497c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:55.321699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e011c2-ef08-4f0b-ad2d-81c4f754497c admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:55.322150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e011c2-ef08-4f0b-ad2d-81c4f754497c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:55.322503 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49e011c2-ef08-4f0b-ad2d-81c4f754497c admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:55.323522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e011c2-ef08-4f0b-ad2d-81c4f754497c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:55.325276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49e011c2-ef08-4f0b-ad2d-81c4f754497c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:55.338363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:55.346339 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1069/2123] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:08:55.351478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7521dc45-7c5d-4be1-b36a-bd31b5519f17 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:55.368549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7521dc45-7c5d-4be1-b36a-bd31b5519f17 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:55.375013 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf404105-6410-4c93-b63b-53adff0de9f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:55.376100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf404105-6410-4c93-b63b-53adff0de9f1 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:55.376568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf404105-6410-4c93-b63b-53adff0de9f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:55.376995 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf404105-6410-4c93-b63b-53adff0de9f1 admin admin] PATH_INFO: `/v3/projects/f2853dee364742ef847cc396f1157ebe/users/84772eb6f1904c12ba9ca791d1044bab/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:55.401421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf404105-6410-4c93-b63b-53adff0de9f1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2853dee364742ef847cc396f1157ebe, user_id=84772eb6f1904c12ba9ca791d1044bab, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:55.405181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf404105-6410-4c93-b63b-53adff0de9f1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:55.428301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1055/2124] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:08:55 2022] PUT /identity/v3/projects/f2853dee364742ef847cc396f1157ebe/users/84772eb6f1904c12ba9ca791d1044bab/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:08:55.433172 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e011c2-ef08-4f0b-ad2d-81c4f754497c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:55.448691 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49e011c2-ef08-4f0b-ad2d-81c4f754497c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:55.510611 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4afd13d-b6d5-4b85-ae2e-6baa6a171dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:55.511293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afd13d-b6d5-4b85-ae2e-6baa6a171dad admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:55.511575 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afd13d-b6d5-4b85-ae2e-6baa6a171dad admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:55.511848 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4afd13d-b6d5-4b85-ae2e-6baa6a171dad admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:55.512582 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4afd13d-b6d5-4b85-ae2e-6baa6a171dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:55.514286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4afd13d-b6d5-4b85-ae2e-6baa6a171dad admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:55.529005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1070/2125] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:08:55.530455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf404105-6410-4c93-b63b-53adff0de9f1 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:55.541989 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf404105-6410-4c93-b63b-53adff0de9f1 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:55.547085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdc071-ab27-47b4-b97e-6f1102b1a8b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:55.547997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdc071-ab27-47b4-b97e-6f1102b1a8b6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:55.548273 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdc071-ab27-47b4-b97e-6f1102b1a8b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:55.548553 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7afdc071-ab27-47b4-b97e-6f1102b1a8b6 admin admin] PATH_INFO: `/v3/projects/f2853dee364742ef847cc396f1157ebe/users/84772eb6f1904c12ba9ca791d1044bab/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:55.550847 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afdc071-ab27-47b4-b97e-6f1102b1a8b6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f2853dee364742ef847cc396f1157ebe, user_id=84772eb6f1904c12ba9ca791d1044bab, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:55.553643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7afdc071-ab27-47b4-b97e-6f1102b1a8b6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:55.566613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1056/2126] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:08:55 2022] PUT /identity/v3/projects/f2853dee364742ef847cc396f1157ebe/users/84772eb6f1904c12ba9ca791d1044bab/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:08:55.577511 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f4499-d109-475d-b69a-f4c94893aeb8 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:55.577804 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f4499-d109-475d-b69a-f4c94893aeb8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:55.578105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-175f4499-d109-475d-b69a-f4c94893aeb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:55.611799 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-175f4499-d109-475d-b69a-f4c94893aeb8 None None] MFA Rules not processed for user `84772eb6f1904c12ba9ca791d1044bab`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:55.613482 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-175f4499-d109-475d-b69a-f4c94893aeb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:55.698918 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1071/2127] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:55 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:55.706996 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdc071-ab27-47b4-b97e-6f1102b1a8b6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:55.709597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7afdc071-ab27-47b4-b97e-6f1102b1a8b6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:55.782363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7afdc071-ab27-47b4-b97e-6f1102b1a8b6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:55.786658 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daadb7e9-ae4c-4588-9d64-4334dc258c11 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:55.787344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadb7e9-ae4c-4588-9d64-4334dc258c11 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:55.787675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadb7e9-ae4c-4588-9d64-4334dc258c11 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:55.787964 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daadb7e9-ae4c-4588-9d64-4334dc258c11 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:55.798621 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadb7e9-ae4c-4588-9d64-4334dc258c11 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:55.800280 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daadb7e9-ae4c-4588-9d64-4334dc258c11 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:55.811687 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1057/2128] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:08:55 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:08:55.897107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f4499-d109-475d-b69a-f4c94893aeb8 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:55.908091 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-175f4499-d109-475d-b69a-f4c94893aeb8 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:55.912818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e67af3-48be-49bb-9894-615fe18e36ff service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:55.913481 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e67af3-48be-49bb-9894-615fe18e36ff service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:55.914431 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e67af3-48be-49bb-9894-615fe18e36ff service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:55.914710 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96e67af3-48be-49bb-9894-615fe18e36ff service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:56.000013 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e67af3-48be-49bb-9894-615fe18e36ff service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:56.001417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96e67af3-48be-49bb-9894-615fe18e36ff service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:56.012445 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1072/2129] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:08:55 2022] GET /identity/v3/auth/tokens => generated 3222 bytes in 116 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:08:59.651931 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f0552-cd47-4480-96c5-3d1716441169 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:59.652450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f0552-cd47-4480-96c5-3d1716441169 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:59.652546 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f0552-cd47-4480-96c5-3d1716441169 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:59.680569 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c03f0552-cd47-4480-96c5-3d1716441169 None None] MFA Rules not processed for user `365acac78e864078af1a3db4842bc149`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:08:59.681739 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c03f0552-cd47-4480-96c5-3d1716441169 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:08:59.769365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1058/2130] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:08:59 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:08:59.774784 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e67af3-48be-49bb-9894-615fe18e36ff service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:59.796863 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:59.845322 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96e67af3-48be-49bb-9894-615fe18e36ff service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:59.852558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed8d76-d8ca-4303-a439-57eb64a470fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:59.853341 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed8d76-d8ca-4303-a439-57eb64a470fd admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:59.853701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed8d76-d8ca-4303-a439-57eb64a470fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:59.854104 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54ed8d76-d8ca-4303-a439-57eb64a470fd admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:59.854939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed8d76-d8ca-4303-a439-57eb64a470fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:59.857065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54ed8d76-d8ca-4303-a439-57eb64a470fd admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:59.886693 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1073/2131] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:08:59 2022] POST /identity/v3/projects => generated 385 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:08:59.890577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f0552-cd47-4480-96c5-3d1716441169 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:08:59.902564 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f0552-cd47-4480-96c5-3d1716441169 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:08:59.953888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:08:59.965287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b2f09-8b7d-448c-95db-44641ca15b9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:08:59.965913 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b2f09-8b7d-448c-95db-44641ca15b9a admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:08:59.966202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b2f09-8b7d-448c-95db-44641ca15b9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:08:59.966551 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b8b2f09-8b7d-448c-95db-44641ca15b9a admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:08:59.967265 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b2f09-8b7d-448c-95db-44641ca15b9a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:08:59.969180 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b8b2f09-8b7d-448c-95db-44641ca15b9a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:08:59.990375 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1059/2132] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:08:59 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:08:59.994599 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed8d76-d8ca-4303-a439-57eb64a470fd admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:00.006133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54ed8d76-d8ca-4303-a439-57eb64a470fd admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:00.010838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1b9d4-4981-4edc-9a36-d11afedf6084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:00.011518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1b9d4-4981-4edc-9a36-d11afedf6084 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:00.011801 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1b9d4-4981-4edc-9a36-d11afedf6084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:00.012072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c1b9d4-4981-4edc-9a36-d11afedf6084 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:00.012806 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c1b9d4-4981-4edc-9a36-d11afedf6084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:00.014453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c1b9d4-4981-4edc-9a36-d11afedf6084 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:00.026049 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1074/2133] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:08:59 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:09:00.030264 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b2f09-8b7d-448c-95db-44641ca15b9a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:00.041005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b8b2f09-8b7d-448c-95db-44641ca15b9a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:00.045964 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd78e16-f1d6-4fc5-b910-f7caf36c8864 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:00.046918 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd78e16-f1d6-4fc5-b910-f7caf36c8864 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:00.047459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd78e16-f1d6-4fc5-b910-f7caf36c8864 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:00.047765 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abd78e16-f1d6-4fc5-b910-f7caf36c8864 admin admin] PATH_INFO: `/v3/projects/b7d4d56298dc40c9b1fde368ed3f19f4/users/9aed963aa25e4d8faa6f715d74f9dd3f/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:00.067976 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd78e16-f1d6-4fc5-b910-f7caf36c8864 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d4d56298dc40c9b1fde368ed3f19f4, user_id=9aed963aa25e4d8faa6f715d74f9dd3f, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:00.070712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abd78e16-f1d6-4fc5-b910-f7caf36c8864 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:00.090124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1060/2134] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:09:00 2022] PUT /identity/v3/projects/b7d4d56298dc40c9b1fde368ed3f19f4/users/9aed963aa25e4d8faa6f715d74f9dd3f/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 60 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:00.093899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1b9d4-4981-4edc-9a36-d11afedf6084 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:00.107615 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c1b9d4-4981-4edc-9a36-d11afedf6084 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:00.194084 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5371818a-bf4f-497d-9aa1-2e255a69f1ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:00.194976 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371818a-bf4f-497d-9aa1-2e255a69f1ea admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:00.195236 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371818a-bf4f-497d-9aa1-2e255a69f1ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:00.195511 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5371818a-bf4f-497d-9aa1-2e255a69f1ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:00.196247 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371818a-bf4f-497d-9aa1-2e255a69f1ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:00.198090 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5371818a-bf4f-497d-9aa1-2e255a69f1ea admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:00.208631 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1075/2135] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:00 2022] GET /identity/v3/roles => generated 1420 bytes in 115 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:09:00.213170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd78e16-f1d6-4fc5-b910-f7caf36c8864 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:00.227528 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abd78e16-f1d6-4fc5-b910-f7caf36c8864 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:00.232323 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de82952-dff2-4bcf-80bd-b90d784b1318 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:00.233231 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de82952-dff2-4bcf-80bd-b90d784b1318 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:00.233607 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de82952-dff2-4bcf-80bd-b90d784b1318 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:00.233785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de82952-dff2-4bcf-80bd-b90d784b1318 admin admin] PATH_INFO: `/v3/projects/b7d4d56298dc40c9b1fde368ed3f19f4/users/9aed963aa25e4d8faa6f715d74f9dd3f/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:00.235862 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de82952-dff2-4bcf-80bd-b90d784b1318 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b7d4d56298dc40c9b1fde368ed3f19f4, user_id=9aed963aa25e4d8faa6f715d74f9dd3f, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:00.239217 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de82952-dff2-4bcf-80bd-b90d784b1318 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:00.250952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1061/2136] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:09:00 2022] PUT /identity/v3/projects/b7d4d56298dc40c9b1fde368ed3f19f4/users/9aed963aa25e4d8faa6f715d74f9dd3f/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:09:05.574633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47446c96-e15d-47f5-acae-74bb750ec7e1 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:05.575228 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47446c96-e15d-47f5-acae-74bb750ec7e1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:05.575228 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47446c96-e15d-47f5-acae-74bb750ec7e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:05.611702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-47446c96-e15d-47f5-acae-74bb750ec7e1 None None] MFA Rules not processed for user `9aed963aa25e4d8faa6f715d74f9dd3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:05.612967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-47446c96-e15d-47f5-acae-74bb750ec7e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:05.688587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1076/2137] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:05 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:05.701213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de82952-dff2-4bcf-80bd-b90d784b1318 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:05.703003 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1de82952-dff2-4bcf-80bd-b90d784b1318 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:05.765984 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de82952-dff2-4bcf-80bd-b90d784b1318 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:05.772005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a05d3-38fc-47d8-982d-81a63f7b0d57 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f80d59fec4994e689e857ba2087d3a23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:05.772674 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a05d3-38fc-47d8-982d-81a63f7b0d57 service glance] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:05.772971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a05d3-38fc-47d8-982d-81a63f7b0d57 service glance] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:05.773243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4a05d3-38fc-47d8-982d-81a63f7b0d57 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:05.859020 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a05d3-38fc-47d8-982d-81a63f7b0d57 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:05.860492 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4a05d3-38fc-47d8-982d-81a63f7b0d57 service glance] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:05.871227 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1062/2138] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:09:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:09:05.991004 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d1d46-81be-486d-804b-920fbc45cb72 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:05.991305 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d1d46-81be-486d-804b-920fbc45cb72 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:05.991592 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779d1d46-81be-486d-804b-920fbc45cb72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:06.012153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-779d1d46-81be-486d-804b-920fbc45cb72 None None] MFA Rules not processed for user `52076f59f68f4d77a0dadbf7a98d2b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:06.013379 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-779d1d46-81be-486d-804b-920fbc45cb72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:06.026400 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:06.106941 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1077/2139] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 11 14:09:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:06.120821 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a05d3-38fc-47d8-982d-81a63f7b0d57 service glance] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:06.122277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7c4a05d3-38fc-47d8-982d-81a63f7b0d57 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:06.188699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:06.228859 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4a05d3-38fc-47d8-982d-81a63f7b0d57 service glance] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:06.234546 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ac861-2af5-4dbb-a7e6-bc8a97779861 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c1369033a44c1c81d4c1c9ff2bbedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:06.235215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ac861-2af5-4dbb-a7e6-bc8a97779861 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:06.235559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ac861-2af5-4dbb-a7e6-bc8a97779861 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:06.235958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5ac861-2af5-4dbb-a7e6-bc8a97779861 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:06.247004 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ac861-2af5-4dbb-a7e6-bc8a97779861 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:06.248652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5ac861-2af5-4dbb-a7e6-bc8a97779861 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:06.259191 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1063/2140] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 11 14:09:06 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 14:09:09.656129 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d9921-2b76-4473-87b2-7e0d7661683d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:09.656735 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d9921-2b76-4473-87b2-7e0d7661683d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:09.656735 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8d9921-2b76-4473-87b2-7e0d7661683d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:09.682563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1b8d9921-2b76-4473-87b2-7e0d7661683d None None] MFA Rules not processed for user `cf6cc756845148ee8fce9021425cb7cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:09.684213 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1b8d9921-2b76-4473-87b2-7e0d7661683d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:09.758360 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1078/2141] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:09 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:09:15.033439 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ac861-2af5-4dbb-a7e6-bc8a97779861 service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:15.209557 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5ac861-2af5-4dbb-a7e6-bc8a97779861 service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:15.214488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a2c34-a901-4234-98d5-14a05510491c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:15.215395 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a2c34-a901-4234-98d5-14a05510491c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:15.215682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a2c34-a901-4234-98d5-14a05510491c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:15.215956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-245a2c34-a901-4234-98d5-14a05510491c admin admin] PATH_INFO: `/v3/users/b53dc890936b4a5d86adb29a5dc688f2` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:15.217241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245a2c34-a901-4234-98d5-14a05510491c admin admin] RBAC: Authorizing `identity:delete_user(user_id=b53dc890936b4a5d86adb29a5dc688f2)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:15.219533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-245a2c34-a901-4234-98d5-14a05510491c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:15.325627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-245a2c34-a901-4234-98d5-14a05510491c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b53dc890936b4a5d86adb29a5dc688f2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:15.327391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-245a2c34-a901-4234-98d5-14a05510491c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b53dc890936b4a5d86adb29a5dc688f2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:15.347339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-245a2c34-a901-4234-98d5-14a05510491c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b53dc890936b4a5d86adb29a5dc688f2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:15.348541 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-245a2c34-a901-4234-98d5-14a05510491c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b53dc890936b4a5d86adb29a5dc688f2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:15.376418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-245a2c34-a901-4234-98d5-14a05510491c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b53dc890936b4a5d86adb29a5dc688f2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:15.397724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1064/2142] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:09:15 2022] DELETE /identity/v3/users/b53dc890936b4a5d86adb29a5dc688f2 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:15.398268 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e263649-8454-4649-9bd6-3192ac5d4d9f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:15.398807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e263649-8454-4649-9bd6-3192ac5d4d9f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:15.398921 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e263649-8454-4649-9bd6-3192ac5d4d9f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:15.429591 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-6e263649-8454-4649-9bd6-3192ac5d4d9f None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:15.431044 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6e263649-8454-4649-9bd6-3192ac5d4d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:15.529209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1079/2143] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:15.538936 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a2c34-a901-4234-98d5-14a05510491c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:15.540531 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-245a2c34-a901-4234-98d5-14a05510491c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:15.624122 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-245a2c34-a901-4234-98d5-14a05510491c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:15.628444 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ea943-2fea-4a04-9453-d4d57a95107e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:15.629222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ea943-2fea-4a04-9453-d4d57a95107e service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:15.629745 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ea943-2fea-4a04-9453-d4d57a95107e service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:15.630136 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60ea943-2fea-4a04-9453-d4d57a95107e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:15.643808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ea943-2fea-4a04-9453-d4d57a95107e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:15.645273 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a60ea943-2fea-4a04-9453-d4d57a95107e service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:15.669986 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1065/2144] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:09:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:09:16.118217 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e263649-8454-4649-9bd6-3192ac5d4d9f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:16.120206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6e263649-8454-4649-9bd6-3192ac5d4d9f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:16.146636 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e263649-8454-4649-9bd6-3192ac5d4d9f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:16.165721 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa580e-8286-47e6-aaf9-bfa9084d004f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:16.166980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa580e-8286-47e6-aaf9-bfa9084d004f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:16.167862 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa580e-8286-47e6-aaf9-bfa9084d004f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:16.167862 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29aa580e-8286-47e6-aaf9-bfa9084d004f admin admin] PATH_INFO: `/v3/projects/a636ba8ac1f344fb84bd8dbe2d86b6d9` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:16.170213 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa580e-8286-47e6-aaf9-bfa9084d004f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a636ba8ac1f344fb84bd8dbe2d86b6d9)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:16.172550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29aa580e-8286-47e6-aaf9-bfa9084d004f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:16.300431 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-29aa580e-8286-47e6-aaf9-bfa9084d004f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a636ba8ac1f344fb84bd8dbe2d86b6d9', 'request_id': 'req-29aa580e-8286-47e6-aaf9-bfa9084d004f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:16.327347 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-29aa580e-8286-47e6-aaf9-bfa9084d004f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a636ba8ac1f344fb84bd8dbe2d86b6d9', 'request_id': 'req-29aa580e-8286-47e6-aaf9-bfa9084d004f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:16.333447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:16.356046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-29aa580e-8286-47e6-aaf9-bfa9084d004f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a636ba8ac1f344fb84bd8dbe2d86b6d9', 'request_id': 'req-29aa580e-8286-47e6-aaf9-bfa9084d004f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:16.394419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-29aa580e-8286-47e6-aaf9-bfa9084d004f admin admin] The token cache is being invalidate because project a636ba8ac1f344fb84bd8dbe2d86b6d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:09:16.394883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-29aa580e-8286-47e6-aaf9-bfa9084d004f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:16.396347 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-29aa580e-8286-47e6-aaf9-bfa9084d004f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:16.399885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1080/2145] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:09:16 2022] DELETE /identity/v3/projects/a636ba8ac1f344fb84bd8dbe2d86b6d9 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:16.404033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ea943-2fea-4a04-9453-d4d57a95107e service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:16.422119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60ea943-2fea-4a04-9453-d4d57a95107e service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:16.430755 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:16.507130 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20fd5372-d6a7-4238-9770-4a3b2a008552 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:16.507896 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd5372-d6a7-4238-9770-4a3b2a008552 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:16.508184 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd5372-d6a7-4238-9770-4a3b2a008552 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:16.508463 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20fd5372-d6a7-4238-9770-4a3b2a008552 admin admin] PATH_INFO: `/v3/users/84772eb6f1904c12ba9ca791d1044bab` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:16.512521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd5372-d6a7-4238-9770-4a3b2a008552 admin admin] RBAC: Authorizing `identity:delete_user(user_id=84772eb6f1904c12ba9ca791d1044bab)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:16.514574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20fd5372-d6a7-4238-9770-4a3b2a008552 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:16.656330 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-20fd5372-d6a7-4238-9770-4a3b2a008552 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '84772eb6f1904c12ba9ca791d1044bab'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:16.657084 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-20fd5372-d6a7-4238-9770-4a3b2a008552 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '84772eb6f1904c12ba9ca791d1044bab'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:16.671562 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-20fd5372-d6a7-4238-9770-4a3b2a008552 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '84772eb6f1904c12ba9ca791d1044bab'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:16.673208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-20fd5372-d6a7-4238-9770-4a3b2a008552 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '84772eb6f1904c12ba9ca791d1044bab'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:16.698567 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-20fd5372-d6a7-4238-9770-4a3b2a008552 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '84772eb6f1904c12ba9ca791d1044bab'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:16.717352 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1066/2146] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:09:16 2022] DELETE /identity/v3/users/84772eb6f1904c12ba9ca791d1044bab => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:17.160269 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa580e-8286-47e6-aaf9-bfa9084d004f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:17.161835 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-29aa580e-8286-47e6-aaf9-bfa9084d004f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:17.186606 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29aa580e-8286-47e6-aaf9-bfa9084d004f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:17.301769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba46381-7ac0-45e9-832e-3475d844399e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:17.306191 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba46381-7ac0-45e9-832e-3475d844399e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:17.307104 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba46381-7ac0-45e9-832e-3475d844399e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:17.307855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba46381-7ac0-45e9-832e-3475d844399e admin admin] PATH_INFO: `/v3/projects/f2853dee364742ef847cc396f1157ebe` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:17.310498 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba46381-7ac0-45e9-832e-3475d844399e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f2853dee364742ef847cc396f1157ebe)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:17.316179 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cba46381-7ac0-45e9-832e-3475d844399e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:17.404822 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cba46381-7ac0-45e9-832e-3475d844399e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f2853dee364742ef847cc396f1157ebe', 'request_id': 'req-cba46381-7ac0-45e9-832e-3475d844399e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:17.415692 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cba46381-7ac0-45e9-832e-3475d844399e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f2853dee364742ef847cc396f1157ebe', 'request_id': 'req-cba46381-7ac0-45e9-832e-3475d844399e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:17.430687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cba46381-7ac0-45e9-832e-3475d844399e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f2853dee364742ef847cc396f1157ebe', 'request_id': 'req-cba46381-7ac0-45e9-832e-3475d844399e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:17.448937 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cba46381-7ac0-45e9-832e-3475d844399e admin admin] The token cache is being invalidate because project f2853dee364742ef847cc396f1157ebe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:09:17.449623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cba46381-7ac0-45e9-832e-3475d844399e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:17.450535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cba46381-7ac0-45e9-832e-3475d844399e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:17.455724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1081/2147] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:09:17 2022] DELETE /identity/v3/projects/f2853dee364742ef847cc396f1157ebe => generated 0 bytes in 293 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:17.457646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518abc0-5fdf-419a-91b1-bb8dffd0cc74 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:17.458182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518abc0-5fdf-419a-91b1-bb8dffd0cc74 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:17.458278 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3518abc0-5fdf-419a-91b1-bb8dffd0cc74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:17.480754 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3518abc0-5fdf-419a-91b1-bb8dffd0cc74 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:17.482020 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3518abc0-5fdf-419a-91b1-bb8dffd0cc74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:17.559144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1067/2148] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:17.569403 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77d140-09b4-451e-8e32-4a9e3320464f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:17.569698 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77d140-09b4-451e-8e32-4a9e3320464f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:17.570004 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b77d140-09b4-451e-8e32-4a9e3320464f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:17.607883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-2b77d140-09b4-451e-8e32-4a9e3320464f None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:17.609657 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2b77d140-09b4-451e-8e32-4a9e3320464f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:17.621680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1082/2149] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:17.623391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518abc0-5fdf-419a-91b1-bb8dffd0cc74 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:17.637978 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3518abc0-5fdf-419a-91b1-bb8dffd0cc74 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:17.644128 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778134c-2f0c-4cec-b27b-769fbe5d78f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:17.644877 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778134c-2f0c-4cec-b27b-769fbe5d78f4 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:17.653346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778134c-2f0c-4cec-b27b-769fbe5d78f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:17.653758 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4778134c-2f0c-4cec-b27b-769fbe5d78f4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:17.654929 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778134c-2f0c-4cec-b27b-769fbe5d78f4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:09:17.655266 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778134c-2f0c-4cec-b27b-769fbe5d78f4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:17.657009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4778134c-2f0c-4cec-b27b-769fbe5d78f4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:17.671515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1068/2150] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:09:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:09:17.676977 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77d140-09b4-451e-8e32-4a9e3320464f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:17.696331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b77d140-09b4-451e-8e32-4a9e3320464f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:17.705423 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fbf94-74fc-4530-ae29-02eaf424e536 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:17.707480 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fbf94-74fc-4530-ae29-02eaf424e536 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:17.708449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fbf94-74fc-4530-ae29-02eaf424e536 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:17.709061 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a96fbf94-74fc-4530-ae29-02eaf424e536 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:17.710495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96fbf94-74fc-4530-ae29-02eaf424e536 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:17.715754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a96fbf94-74fc-4530-ae29-02eaf424e536 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:17.782029 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1083/2151] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:09:17 2022] POST /identity/v3/projects => generated 365 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:09:17.786129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778134c-2f0c-4cec-b27b-769fbe5d78f4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:17.797439 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4778134c-2f0c-4cec-b27b-769fbe5d78f4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:17.820536 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:17.894056 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b29ad4-6260-4f34-92a4-9af9298496aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:17.895082 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b29ad4-6260-4f34-92a4-9af9298496aa admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:17.895384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b29ad4-6260-4f34-92a4-9af9298496aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:17.895662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05b29ad4-6260-4f34-92a4-9af9298496aa admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:17.896682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b29ad4-6260-4f34-92a4-9af9298496aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:17.899021 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05b29ad4-6260-4f34-92a4-9af9298496aa admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:17.925786 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1069/2152] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:09:17 2022] POST /identity/v3/users => generated 283 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:09:17.930270 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fbf94-74fc-4530-ae29-02eaf424e536 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:17.943763 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a96fbf94-74fc-4530-ae29-02eaf424e536 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:17.953342 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86df9e4-c33a-46d6-9820-646c8e7538cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:17.954574 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86df9e4-c33a-46d6-9820-646c8e7538cd admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:17.954959 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86df9e4-c33a-46d6-9820-646c8e7538cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:17.955368 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86df9e4-c33a-46d6-9820-646c8e7538cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:17.956850 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86df9e4-c33a-46d6-9820-646c8e7538cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:17.959152 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86df9e4-c33a-46d6-9820-646c8e7538cd admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:17.974363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1084/2153] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:09:17.978621 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b29ad4-6260-4f34-92a4-9af9298496aa admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:17.989685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05b29ad4-6260-4f34-92a4-9af9298496aa admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:17.995597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567dc0fa-ef7a-4a02-8d8c-07917b89bb31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:17.996437 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567dc0fa-ef7a-4a02-8d8c-07917b89bb31 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:17.996732 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567dc0fa-ef7a-4a02-8d8c-07917b89bb31 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:17.997006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567dc0fa-ef7a-4a02-8d8c-07917b89bb31 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:17.997943 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567dc0fa-ef7a-4a02-8d8c-07917b89bb31 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:17.999619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567dc0fa-ef7a-4a02-8d8c-07917b89bb31 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:18.015886 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1070/2154] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:17 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:09:18.017363 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86df9e4-c33a-46d6-9820-646c8e7538cd admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:18.028431 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86df9e4-c33a-46d6-9820-646c8e7538cd admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:18.034861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b103b-693e-4e75-b8fe-e1b019916ef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:18.035879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b103b-693e-4e75-b8fe-e1b019916ef0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:18.036167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b103b-693e-4e75-b8fe-e1b019916ef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:18.036587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-191b103b-693e-4e75-b8fe-e1b019916ef0 admin admin] PATH_INFO: `/v3/projects/aee33ba170f348cbadee9fdc33280ff9/users/3ad3d497764b43748beeb84eda627458/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:18.057622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b103b-693e-4e75-b8fe-e1b019916ef0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=aee33ba170f348cbadee9fdc33280ff9, user_id=3ad3d497764b43748beeb84eda627458, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:18.060373 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-191b103b-693e-4e75-b8fe-e1b019916ef0 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:18.092633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1085/2155] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:09:18 2022] PUT /identity/v3/projects/aee33ba170f348cbadee9fdc33280ff9/users/3ad3d497764b43748beeb84eda627458/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 67 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:18.094463 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b9ab4-3efc-4502-afa6-f960f1417e02 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:18.094767 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b9ab4-3efc-4502-afa6-f960f1417e02 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:18.095078 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d6b9ab4-3efc-4502-afa6-f960f1417e02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:18.127963 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4d6b9ab4-3efc-4502-afa6-f960f1417e02 None None] MFA Rules not processed for user `3ad3d497764b43748beeb84eda627458`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:18.129421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4d6b9ab4-3efc-4502-afa6-f960f1417e02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:18.210815 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1071/2156] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:18 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:09:18.215452 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b103b-693e-4e75-b8fe-e1b019916ef0 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:18.241197 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=8,keystone:SELECT=89,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:18.305019 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-191b103b-693e-4e75-b8fe-e1b019916ef0 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:18.310997 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad7fe1-893f-4955-9cab-fb61a5c3f103 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:18.311810 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad7fe1-893f-4955-9cab-fb61a5c3f103 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:18.312085 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad7fe1-893f-4955-9cab-fb61a5c3f103 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:18.312370 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ad7fe1-893f-4955-9cab-fb61a5c3f103 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:18.313165 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad7fe1-893f-4955-9cab-fb61a5c3f103 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:18.315546 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6ad7fe1-893f-4955-9cab-fb61a5c3f103 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:18.353510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1086/2157] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:09:18 2022] POST /identity/v3/projects => generated 365 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:09:18.359508 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b9ab4-3efc-4502-afa6-f960f1417e02 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:18.381937 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d6b9ab4-3efc-4502-afa6-f960f1417e02 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:18.467385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec4a5d-a54b-43c7-82e6-08a67a66ca6c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:18.468050 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec4a5d-a54b-43c7-82e6-08a67a66ca6c admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:18.468404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec4a5d-a54b-43c7-82e6-08a67a66ca6c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:18.468852 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26ec4a5d-a54b-43c7-82e6-08a67a66ca6c admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:18.469763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec4a5d-a54b-43c7-82e6-08a67a66ca6c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:18.472586 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26ec4a5d-a54b-43c7-82e6-08a67a66ca6c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:18.507227 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1072/2158] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:09:18 2022] POST /identity/v3/users => generated 289 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:09:18.510853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad7fe1-893f-4955-9cab-fb61a5c3f103 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:18.527284 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6ad7fe1-893f-4955-9cab-fb61a5c3f103 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:18.532235 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e542a3-02f2-4910-9599-e6f114144319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:18.533144 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e542a3-02f2-4910-9599-e6f114144319 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:18.533500 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e542a3-02f2-4910-9599-e6f114144319 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:18.533887 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e542a3-02f2-4910-9599-e6f114144319 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:18.534768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e542a3-02f2-4910-9599-e6f114144319 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:18.538654 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78e542a3-02f2-4910-9599-e6f114144319 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:18.557842 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1087/2159] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:09:18.561720 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec4a5d-a54b-43c7-82e6-08a67a66ca6c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:18.575690 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26ec4a5d-a54b-43c7-82e6-08a67a66ca6c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:18.581609 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7c579-38ea-4644-b95c-2bb7e6101c0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:18.582550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7c579-38ea-4644-b95c-2bb7e6101c0d admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:18.582839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7c579-38ea-4644-b95c-2bb7e6101c0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:18.583148 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7c579-38ea-4644-b95c-2bb7e6101c0d admin admin] PATH_INFO: `/v3/projects/64506ec559cf4200ad356e13ecfbc78c/users/7bfee10bb3594de690f9e19af5b0e598/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:18.598557 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:18.605774 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa7c579-38ea-4644-b95c-2bb7e6101c0d admin admin] RBAC: Authorizing `identity:create_grant(project_id=64506ec559cf4200ad356e13ecfbc78c, user_id=7bfee10bb3594de690f9e19af5b0e598, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:18.607547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa7c579-38ea-4644-b95c-2bb7e6101c0d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:18.627986 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1073/2160] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:09:18 2022] PUT /identity/v3/projects/64506ec559cf4200ad356e13ecfbc78c/users/7bfee10bb3594de690f9e19af5b0e598/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 66 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:18.631803 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e542a3-02f2-4910-9599-e6f114144319 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:18.646292 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e542a3-02f2-4910-9599-e6f114144319 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:18.724482 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ca239d-283b-464e-b40c-05765ea5f90a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:18.724837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca239d-283b-464e-b40c-05765ea5f90a admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:18.725118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca239d-283b-464e-b40c-05765ea5f90a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:18.725377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ca239d-283b-464e-b40c-05765ea5f90a admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:18.726109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca239d-283b-464e-b40c-05765ea5f90a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:18.727904 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17ca239d-283b-464e-b40c-05765ea5f90a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:18.740314 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1088/2161] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:18 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:09:18.743988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7c579-38ea-4644-b95c-2bb7e6101c0d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:18.758170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7c579-38ea-4644-b95c-2bb7e6101c0d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:18.764345 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7599e1e-fa25-417d-ba3b-b8ae69529bda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:18.766881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7599e1e-fa25-417d-ba3b-b8ae69529bda admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:18.767284 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7599e1e-fa25-417d-ba3b-b8ae69529bda admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:18.767674 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7599e1e-fa25-417d-ba3b-b8ae69529bda admin admin] PATH_INFO: `/v3/projects/64506ec559cf4200ad356e13ecfbc78c/users/7bfee10bb3594de690f9e19af5b0e598/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:18.769967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7599e1e-fa25-417d-ba3b-b8ae69529bda admin admin] RBAC: Authorizing `identity:create_grant(project_id=64506ec559cf4200ad356e13ecfbc78c, user_id=7bfee10bb3594de690f9e19af5b0e598, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:18.772768 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7599e1e-fa25-417d-ba3b-b8ae69529bda admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:18.786221 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1074/2162] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:09:18 2022] PUT /identity/v3/projects/64506ec559cf4200ad356e13ecfbc78c/users/7bfee10bb3594de690f9e19af5b0e598/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 43 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:18.797813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d0aee-d08c-4e5f-97e3-cbc7018c7d35 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:18.798109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d0aee-d08c-4e5f-97e3-cbc7018c7d35 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:18.798471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1d0aee-d08c-4e5f-97e3-cbc7018c7d35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:18.838851 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-aa1d0aee-d08c-4e5f-97e3-cbc7018c7d35 None None] MFA Rules not processed for user `7bfee10bb3594de690f9e19af5b0e598`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:18.840410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-aa1d0aee-d08c-4e5f-97e3-cbc7018c7d35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:18.948030 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1089/2163] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:18 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:18.960726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7599e1e-fa25-417d-ba3b-b8ae69529bda admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:18.962567 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f7599e1e-fa25-417d-ba3b-b8ae69529bda admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:19.046537 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7599e1e-fa25-417d-ba3b-b8ae69529bda admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:19.065994 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55914679-28ef-4bff-9900-c276e77e4633 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:19.067028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55914679-28ef-4bff-9900-c276e77e4633 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:19.067485 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55914679-28ef-4bff-9900-c276e77e4633 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:19.067939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55914679-28ef-4bff-9900-c276e77e4633 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:19.089952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55914679-28ef-4bff-9900-c276e77e4633 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:19.091725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55914679-28ef-4bff-9900-c276e77e4633 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:19.105550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1075/2164] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:09:18 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:09:19.220687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d0aee-d08c-4e5f-97e3-cbc7018c7d35 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:19.231327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1d0aee-d08c-4e5f-97e3-cbc7018c7d35 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:19.235797 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f90b73f-bcd8-4a13-937b-bde4bbf6342d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:19.236408 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90b73f-bcd8-4a13-937b-bde4bbf6342d service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:19.241834 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90b73f-bcd8-4a13-937b-bde4bbf6342d service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:19.242211 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f90b73f-bcd8-4a13-937b-bde4bbf6342d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:19.253560 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:19.313539 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f90b73f-bcd8-4a13-937b-bde4bbf6342d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:19.316238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f90b73f-bcd8-4a13-937b-bde4bbf6342d service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:19.326445 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1090/2165] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:09:19 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:09:20.999334 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14bf30-128d-420b-b1c5-bf6a8e8d737c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:20.999874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14bf30-128d-420b-b1c5-bf6a8e8d737c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:20.999874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f14bf30-128d-420b-b1c5-bf6a8e8d737c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:21.019206 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-9f14bf30-128d-420b-b1c5-bf6a8e8d737c None None] MFA Rules not processed for user `365acac78e864078af1a3db4842bc149`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:21.020432 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9f14bf30-128d-420b-b1c5-bf6a8e8d737c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:21.081062 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1076/2166] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:27.990105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474efce-b728-428a-b1a1-4f35eb6658cc None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:27.990739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474efce-b728-428a-b1a1-4f35eb6658cc None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:27.990739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8474efce-b728-428a-b1a1-4f35eb6658cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:28.010171 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8474efce-b728-428a-b1a1-4f35eb6658cc None None] MFA Rules not processed for user `52076f59f68f4d77a0dadbf7a98d2b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:28.012051 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8474efce-b728-428a-b1a1-4f35eb6658cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:28.099096 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1091/2167] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 11 14:09:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:28.135841 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14bf30-128d-420b-b1c5-bf6a8e8d737c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:28.138422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9f14bf30-128d-420b-b1c5-bf6a8e8d737c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:28.270904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f14bf30-128d-420b-b1c5-bf6a8e8d737c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:28.280849 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a824221b-5654-4915-8ecc-77434cfbb119 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c1369033a44c1c81d4c1c9ff2bbedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:28.281307 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824221b-5654-4915-8ecc-77434cfbb119 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:28.281726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824221b-5654-4915-8ecc-77434cfbb119 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:28.282111 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a824221b-5654-4915-8ecc-77434cfbb119 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:28.297520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824221b-5654-4915-8ecc-77434cfbb119 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:28.300458 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a824221b-5654-4915-8ecc-77434cfbb119 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:28.318653 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1077/2168] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 11 14:09:28 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 11 14:09:31.297284 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474efce-b728-428a-b1a1-4f35eb6658cc None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:31.435530 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8474efce-b728-428a-b1a1-4f35eb6658cc None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:31.440654 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c538ba-cb08-4d4f-a992-cf0c2bf88bf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:31.441657 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c538ba-cb08-4d4f-a992-cf0c2bf88bf7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:31.441964 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c538ba-cb08-4d4f-a992-cf0c2bf88bf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:31.442248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6c538ba-cb08-4d4f-a992-cf0c2bf88bf7 admin admin] PATH_INFO: `/v3/users/3ad3d497764b43748beeb84eda627458` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:31.452205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c538ba-cb08-4d4f-a992-cf0c2bf88bf7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3ad3d497764b43748beeb84eda627458)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:31.454283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6c538ba-cb08-4d4f-a992-cf0c2bf88bf7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:31.542615 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a6c538ba-cb08-4d4f-a992-cf0c2bf88bf7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3ad3d497764b43748beeb84eda627458'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:31.586492 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=8 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:31.594840 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a6c538ba-cb08-4d4f-a992-cf0c2bf88bf7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3ad3d497764b43748beeb84eda627458'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:31.605201 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a6c538ba-cb08-4d4f-a992-cf0c2bf88bf7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3ad3d497764b43748beeb84eda627458'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:31.607054 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a6c538ba-cb08-4d4f-a992-cf0c2bf88bf7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3ad3d497764b43748beeb84eda627458'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:31.621038 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a6c538ba-cb08-4d4f-a992-cf0c2bf88bf7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3ad3d497764b43748beeb84eda627458'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:31.632281 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1092/2169] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:09:31 2022] DELETE /identity/v3/users/3ad3d497764b43748beeb84eda627458 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:09:31.636604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7cbf7-634d-47a3-b592-8c91ff068b4e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:31.637063 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7cbf7-634d-47a3-b592-8c91ff068b4e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:31.637179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8f7cbf7-634d-47a3-b592-8c91ff068b4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:31.663546 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e8f7cbf7-634d-47a3-b592-8c91ff068b4e None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:31.664907 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e8f7cbf7-634d-47a3-b592-8c91ff068b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:31.674522 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:31.742726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1078/2170] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:31.752841 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c538ba-cb08-4d4f-a992-cf0c2bf88bf7 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:31.754834 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a6c538ba-cb08-4d4f-a992-cf0c2bf88bf7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:31.829119 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6c538ba-cb08-4d4f-a992-cf0c2bf88bf7 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:31.834173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf25d1-3fe7-49a4-b11b-6c8c44eec873 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:31.835205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf25d1-3fe7-49a4-b11b-6c8c44eec873 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:31.835576 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf25d1-3fe7-49a4-b11b-6c8c44eec873 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:31.835930 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bdf25d1-3fe7-49a4-b11b-6c8c44eec873 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:31.850947 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf25d1-3fe7-49a4-b11b-6c8c44eec873 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:31.853654 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bdf25d1-3fe7-49a4-b11b-6c8c44eec873 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:31.869761 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1093/2171] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:09:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 117 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:09:32.303617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7cbf7-634d-47a3-b592-8c91ff068b4e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:32.305772 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e8f7cbf7-634d-47a3-b592-8c91ff068b4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:32.328216 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8f7cbf7-634d-47a3-b592-8c91ff068b4e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:32.334801 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12fc8e-e6eb-43a8-8537-f57fb66174fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:32.335747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12fc8e-e6eb-43a8-8537-f57fb66174fc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:32.336029 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12fc8e-e6eb-43a8-8537-f57fb66174fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:32.336304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12fc8e-e6eb-43a8-8537-f57fb66174fc admin admin] PATH_INFO: `/v3/projects/aee33ba170f348cbadee9fdc33280ff9` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:32.337925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12fc8e-e6eb-43a8-8537-f57fb66174fc admin admin] RBAC: Authorizing `identity:delete_project(project_id=aee33ba170f348cbadee9fdc33280ff9)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:32.339877 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12fc8e-e6eb-43a8-8537-f57fb66174fc admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:32.443910 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8f12fc8e-e6eb-43a8-8537-f57fb66174fc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aee33ba170f348cbadee9fdc33280ff9', 'request_id': 'req-8f12fc8e-e6eb-43a8-8537-f57fb66174fc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:32.451814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8f12fc8e-e6eb-43a8-8537-f57fb66174fc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aee33ba170f348cbadee9fdc33280ff9', 'request_id': 'req-8f12fc8e-e6eb-43a8-8537-f57fb66174fc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:32.463926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8f12fc8e-e6eb-43a8-8537-f57fb66174fc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aee33ba170f348cbadee9fdc33280ff9', 'request_id': 'req-8f12fc8e-e6eb-43a8-8537-f57fb66174fc'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:32.479930 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8f12fc8e-e6eb-43a8-8537-f57fb66174fc admin admin] The token cache is being invalidate because project aee33ba170f348cbadee9fdc33280ff9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:09:32.480170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8f12fc8e-e6eb-43a8-8537-f57fb66174fc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:32.494523 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8f12fc8e-e6eb-43a8-8537-f57fb66174fc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:32.498513 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1079/2172] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:09:32 2022] DELETE /identity/v3/projects/aee33ba170f348cbadee9fdc33280ff9 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:32.500769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf25d1-3fe7-49a4-b11b-6c8c44eec873 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:32.511986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bdf25d1-3fe7-49a4-b11b-6c8c44eec873 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:32.595304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1a57598-762f-4f94-8e6e-5f8603be631e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:32.596860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a57598-762f-4f94-8e6e-5f8603be631e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:32.597348 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a57598-762f-4f94-8e6e-5f8603be631e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:32.597792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1a57598-762f-4f94-8e6e-5f8603be631e admin admin] PATH_INFO: `/v3/users/7bfee10bb3594de690f9e19af5b0e598` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:32.599387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a57598-762f-4f94-8e6e-5f8603be631e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7bfee10bb3594de690f9e19af5b0e598)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:32.601601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1a57598-762f-4f94-8e6e-5f8603be631e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:32.675864 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a1a57598-762f-4f94-8e6e-5f8603be631e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7bfee10bb3594de690f9e19af5b0e598'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:32.700733 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a1a57598-762f-4f94-8e6e-5f8603be631e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7bfee10bb3594de690f9e19af5b0e598'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:32.714572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a1a57598-762f-4f94-8e6e-5f8603be631e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7bfee10bb3594de690f9e19af5b0e598'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:32.715340 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a1a57598-762f-4f94-8e6e-5f8603be631e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7bfee10bb3594de690f9e19af5b0e598'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:32.723127 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:32.727045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a1a57598-762f-4f94-8e6e-5f8603be631e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7bfee10bb3594de690f9e19af5b0e598'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:32.737806 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1094/2173] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:09:32 2022] DELETE /identity/v3/users/7bfee10bb3594de690f9e19af5b0e598 => generated 0 bytes in 238 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:33.186099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12fc8e-e6eb-43a8-8537-f57fb66174fc admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:33.188560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8f12fc8e-e6eb-43a8-8537-f57fb66174fc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:33.213567 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12fc8e-e6eb-43a8-8537-f57fb66174fc admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:33.285101 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3801ccc9-2ed9-47e9-ac68-d68eae5f667d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:33.286279 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801ccc9-2ed9-47e9-ac68-d68eae5f667d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:33.286600 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801ccc9-2ed9-47e9-ac68-d68eae5f667d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:33.286877 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3801ccc9-2ed9-47e9-ac68-d68eae5f667d admin admin] PATH_INFO: `/v3/projects/64506ec559cf4200ad356e13ecfbc78c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:33.288126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801ccc9-2ed9-47e9-ac68-d68eae5f667d admin admin] RBAC: Authorizing `identity:delete_project(project_id=64506ec559cf4200ad356e13ecfbc78c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:33.291186 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3801ccc9-2ed9-47e9-ac68-d68eae5f667d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:33.339717 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:33.348944 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3801ccc9-2ed9-47e9-ac68-d68eae5f667d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64506ec559cf4200ad356e13ecfbc78c', 'request_id': 'req-3801ccc9-2ed9-47e9-ac68-d68eae5f667d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:33.357266 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3801ccc9-2ed9-47e9-ac68-d68eae5f667d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64506ec559cf4200ad356e13ecfbc78c', 'request_id': 'req-3801ccc9-2ed9-47e9-ac68-d68eae5f667d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:33.369179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3801ccc9-2ed9-47e9-ac68-d68eae5f667d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64506ec559cf4200ad356e13ecfbc78c', 'request_id': 'req-3801ccc9-2ed9-47e9-ac68-d68eae5f667d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:33.385605 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3801ccc9-2ed9-47e9-ac68-d68eae5f667d admin admin] The token cache is being invalidate because project 64506ec559cf4200ad356e13ecfbc78c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:09:33.385904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3801ccc9-2ed9-47e9-ac68-d68eae5f667d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:33.386602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3801ccc9-2ed9-47e9-ac68-d68eae5f667d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:33.388185 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1080/2174] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:09:33 2022] DELETE /identity/v3/projects/64506ec559cf4200ad356e13ecfbc78c => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:33.394425 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50c9bc-0a84-4cff-94c8-5e8438e6d078 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:33.394850 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50c9bc-0a84-4cff-94c8-5e8438e6d078 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:33.395170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c50c9bc-0a84-4cff-94c8-5e8438e6d078 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:33.416546 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-0c50c9bc-0a84-4cff-94c8-5e8438e6d078 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:33.417859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0c50c9bc-0a84-4cff-94c8-5e8438e6d078 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:33.481572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1095/2175] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:33.489533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801a1b6-9f17-48b3-a36c-ebadb1ab3d13 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:33.489845 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801a1b6-9f17-48b3-a36c-ebadb1ab3d13 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:33.490128 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d801a1b6-9f17-48b3-a36c-ebadb1ab3d13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:33.510597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d801a1b6-9f17-48b3-a36c-ebadb1ab3d13 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:33.511825 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d801a1b6-9f17-48b3-a36c-ebadb1ab3d13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:33.541212 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1081/2176] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 53 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:09:33.546034 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50c9bc-0a84-4cff-94c8-5e8438e6d078 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:33.556344 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c50c9bc-0a84-4cff-94c8-5e8438e6d078 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:33.561057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4846c7-982f-42e3-815d-4bdf1d1c2b09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:33.561721 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4846c7-982f-42e3-815d-4bdf1d1c2b09 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:33.562014 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4846c7-982f-42e3-815d-4bdf1d1c2b09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:33.562293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c4846c7-982f-42e3-815d-4bdf1d1c2b09 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:33.563075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4846c7-982f-42e3-815d-4bdf1d1c2b09 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:09:33.563390 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4846c7-982f-42e3-815d-4bdf1d1c2b09 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:33.565083 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c4846c7-982f-42e3-815d-4bdf1d1c2b09 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:33.578370 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1096/2177] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:09:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:09:33.584611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801a1b6-9f17-48b3-a36c-ebadb1ab3d13 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:33.594907 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d801a1b6-9f17-48b3-a36c-ebadb1ab3d13 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:33.600057 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b27ced-0da0-46da-b446-2993373474f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:33.600740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b27ced-0da0-46da-b446-2993373474f3 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:33.601028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b27ced-0da0-46da-b446-2993373474f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:33.601308 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b27ced-0da0-46da-b446-2993373474f3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:33.602028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b27ced-0da0-46da-b446-2993373474f3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:33.603999 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b27ced-0da0-46da-b446-2993373474f3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:33.648268 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1082/2178] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:09:33 2022] POST /identity/v3/projects => generated 391 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:09:33.649798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4846c7-982f-42e3-815d-4bdf1d1c2b09 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:33.660449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c4846c7-982f-42e3-815d-4bdf1d1c2b09 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:33.750161 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5d4c7-68e0-439a-8cd6-8d67da621784 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:33.750161 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5d4c7-68e0-439a-8cd6-8d67da621784 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:33.750161 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5d4c7-68e0-439a-8cd6-8d67da621784 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:33.750161 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae5d4c7-68e0-439a-8cd6-8d67da621784 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:33.750918 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5d4c7-68e0-439a-8cd6-8d67da621784 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:33.751123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae5d4c7-68e0-439a-8cd6-8d67da621784 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:33.783072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1097/2179] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:09:33 2022] POST /identity/v3/users => generated 296 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:09:33.787726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b27ced-0da0-46da-b446-2993373474f3 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:33.802750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b27ced-0da0-46da-b446-2993373474f3 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:33.810675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfd152-5e62-4c8a-8a87-beada5b3aa18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:33.811489 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfd152-5e62-4c8a-8a87-beada5b3aa18 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:33.811797 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfd152-5e62-4c8a-8a87-beada5b3aa18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:33.812265 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfbfd152-5e62-4c8a-8a87-beada5b3aa18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:33.813075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfd152-5e62-4c8a-8a87-beada5b3aa18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:33.815067 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfbfd152-5e62-4c8a-8a87-beada5b3aa18 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:33.827263 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1083/2180] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:09:33.832186 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5d4c7-68e0-439a-8cd6-8d67da621784 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:33.849796 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae5d4c7-68e0-439a-8cd6-8d67da621784 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:33.861543 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a71d1d2-0578-4cde-b94b-765951e24393 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:33.864730 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71d1d2-0578-4cde-b94b-765951e24393 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:33.865025 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71d1d2-0578-4cde-b94b-765951e24393 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:33.865484 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a71d1d2-0578-4cde-b94b-765951e24393 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:33.866252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a71d1d2-0578-4cde-b94b-765951e24393 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:33.868384 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a71d1d2-0578-4cde-b94b-765951e24393 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:33.892703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1098/2181] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:33 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:09:33.897186 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfd152-5e62-4c8a-8a87-beada5b3aa18 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:33.907955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfbfd152-5e62-4c8a-8a87-beada5b3aa18 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:33.914272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee745e-8396-48a3-9b25-d3147473b396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:33.915229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee745e-8396-48a3-9b25-d3147473b396 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:33.915491 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee745e-8396-48a3-9b25-d3147473b396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:33.915768 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ee745e-8396-48a3-9b25-d3147473b396 admin admin] PATH_INFO: `/v3/projects/e94edc8de70a4c5d8197b6cdfd3238f3/users/fc671d6fc7a645ce9f6356f30e318f7a/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:33.937736 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee745e-8396-48a3-9b25-d3147473b396 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e94edc8de70a4c5d8197b6cdfd3238f3, user_id=fc671d6fc7a645ce9f6356f30e318f7a, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:33.941400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ee745e-8396-48a3-9b25-d3147473b396 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:33.961055 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1084/2182] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:09:33 2022] PUT /identity/v3/projects/e94edc8de70a4c5d8197b6cdfd3238f3/users/fc671d6fc7a645ce9f6356f30e318f7a/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:33.972252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541104e-0197-44b1-aa6e-0aba7938842e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:33.972624 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541104e-0197-44b1-aa6e-0aba7938842e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:33.972908 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6541104e-0197-44b1-aa6e-0aba7938842e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:34.019729 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-6541104e-0197-44b1-aa6e-0aba7938842e None None] MFA Rules not processed for user `fc671d6fc7a645ce9f6356f30e318f7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:34.021398 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6541104e-0197-44b1-aa6e-0aba7938842e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:34.106908 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:34.129123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1099/2183] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:33 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:34.134013 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee745e-8396-48a3-9b25-d3147473b396 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:34.219588 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ee745e-8396-48a3-9b25-d3147473b396 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:34.225372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd2aa8-aca5-40d8-a11c-000e42c9b0c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:34.225372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd2aa8-aca5-40d8-a11c-000e42c9b0c8 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:34.225692 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd2aa8-aca5-40d8-a11c-000e42c9b0c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:34.225837 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cbd2aa8-aca5-40d8-a11c-000e42c9b0c8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:34.226583 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd2aa8-aca5-40d8-a11c-000e42c9b0c8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:34.230345 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cbd2aa8-aca5-40d8-a11c-000e42c9b0c8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:34.258954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1085/2184] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:09:34 2022] POST /identity/v3/projects => generated 391 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:09:34.263165 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6541104e-0197-44b1-aa6e-0aba7938842e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:34.275167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6541104e-0197-44b1-aa6e-0aba7938842e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:34.334828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0889bb-af74-468b-99d9-f8d85821baf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:34.335611 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0889bb-af74-468b-99d9-f8d85821baf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:34.335952 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0889bb-af74-468b-99d9-f8d85821baf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:34.336270 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0889bb-af74-468b-99d9-f8d85821baf9 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:34.337014 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0889bb-af74-468b-99d9-f8d85821baf9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:34.339183 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0889bb-af74-468b-99d9-f8d85821baf9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:34.361136 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1100/2185] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:09:34 2022] POST /identity/v3/users => generated 302 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:09:34.365473 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd2aa8-aca5-40d8-a11c-000e42c9b0c8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:34.376935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cbd2aa8-aca5-40d8-a11c-000e42c9b0c8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:34.382264 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e20375-e3c3-47a3-b111-1b061fd893f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:34.382966 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e20375-e3c3-47a3-b111-1b061fd893f1 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:34.383248 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e20375-e3c3-47a3-b111-1b061fd893f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:34.383526 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90e20375-e3c3-47a3-b111-1b061fd893f1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:34.384264 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e20375-e3c3-47a3-b111-1b061fd893f1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:34.385864 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90e20375-e3c3-47a3-b111-1b061fd893f1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:34.396735 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1086/2186] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:09:34.401143 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0889bb-af74-468b-99d9-f8d85821baf9 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:34.411839 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0889bb-af74-468b-99d9-f8d85821baf9 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:34.417196 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49c674-5211-4ac7-a109-f4920b1617d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:34.418349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49c674-5211-4ac7-a109-f4920b1617d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:34.418807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49c674-5211-4ac7-a109-f4920b1617d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:34.419319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e49c674-5211-4ac7-a109-f4920b1617d4 admin admin] PATH_INFO: `/v3/projects/2784d06d34e448edb81bacb230fbed55/users/27770228305b454fa004d5e78883ffd7/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:34.436570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49c674-5211-4ac7-a109-f4920b1617d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2784d06d34e448edb81bacb230fbed55, user_id=27770228305b454fa004d5e78883ffd7, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:34.439418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e49c674-5211-4ac7-a109-f4920b1617d4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:34.457742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1101/2187] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:09:34 2022] PUT /identity/v3/projects/2784d06d34e448edb81bacb230fbed55/users/27770228305b454fa004d5e78883ffd7/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:09:34.461893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e20375-e3c3-47a3-b111-1b061fd893f1 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:34.471844 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90e20375-e3c3-47a3-b111-1b061fd893f1 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:34.508795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:34.535759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e41f1b6-6e55-4d0c-b9d1-9b8af15d95ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:34.536417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41f1b6-6e55-4d0c-b9d1-9b8af15d95ea admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:34.536706 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41f1b6-6e55-4d0c-b9d1-9b8af15d95ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:34.536990 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e41f1b6-6e55-4d0c-b9d1-9b8af15d95ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:34.537732 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41f1b6-6e55-4d0c-b9d1-9b8af15d95ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:34.539403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e41f1b6-6e55-4d0c-b9d1-9b8af15d95ea admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:34.551132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1087/2188] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:34 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:09:34.555641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49c674-5211-4ac7-a109-f4920b1617d4 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:34.567049 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e49c674-5211-4ac7-a109-f4920b1617d4 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:34.571754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b817d-b08f-4ed0-afc3-bad192b522f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:34.572669 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b817d-b08f-4ed0-afc3-bad192b522f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:34.572951 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b817d-b08f-4ed0-afc3-bad192b522f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:34.573230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b4b817d-b08f-4ed0-afc3-bad192b522f0 admin admin] PATH_INFO: `/v3/projects/2784d06d34e448edb81bacb230fbed55/users/27770228305b454fa004d5e78883ffd7/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:34.575267 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b817d-b08f-4ed0-afc3-bad192b522f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2784d06d34e448edb81bacb230fbed55, user_id=27770228305b454fa004d5e78883ffd7, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:34.577902 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b4b817d-b08f-4ed0-afc3-bad192b522f0 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:34.588383 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1102/2189] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:09:34 2022] PUT /identity/v3/projects/2784d06d34e448edb81bacb230fbed55/users/27770228305b454fa004d5e78883ffd7/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:34.599606 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e415be-465a-4dc4-b8ca-503fc593d810 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:34.599902 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e415be-465a-4dc4-b8ca-503fc593d810 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:34.600185 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e415be-465a-4dc4-b8ca-503fc593d810 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:34.631176 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-17e415be-465a-4dc4-b8ca-503fc593d810 None None] MFA Rules not processed for user `27770228305b454fa004d5e78883ffd7`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:34.632422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-17e415be-465a-4dc4-b8ca-503fc593d810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:34.712015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1088/2190] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:34 2022] POST /identity/v3/auth/tokens => generated 3325 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:34.720259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b817d-b08f-4ed0-afc3-bad192b522f0 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:34.722685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4b4b817d-b08f-4ed0-afc3-bad192b522f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:34.787141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b4b817d-b08f-4ed0-afc3-bad192b522f0 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:34.791622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae9d9d-0acc-4cc7-9861-8938d53d8362 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:34.792294 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae9d9d-0acc-4cc7-9861-8938d53d8362 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:34.792587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae9d9d-0acc-4cc7-9861-8938d53d8362 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:34.793112 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ae9d9d-0acc-4cc7-9861-8938d53d8362 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:34.805847 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae9d9d-0acc-4cc7-9861-8938d53d8362 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:34.807672 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ae9d9d-0acc-4cc7-9861-8938d53d8362 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:34.820504 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1103/2191] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:09:34 2022] GET /identity/v3/auth/tokens => generated 3325 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:09:34.871568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e415be-465a-4dc4-b8ca-503fc593d810 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:34.872980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-17e415be-465a-4dc4-b8ca-503fc593d810 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:34.893103 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e415be-465a-4dc4-b8ca-503fc593d810 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:34.912456 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff3804b-5e51-4299-be99-8f859cf6c7bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:34.913163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3804b-5e51-4299-be99-8f859cf6c7bb service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:34.913466 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3804b-5e51-4299-be99-8f859cf6c7bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:34.913721 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff3804b-5e51-4299-be99-8f859cf6c7bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:34.980681 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3804b-5e51-4299-be99-8f859cf6c7bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:34.982184 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff3804b-5e51-4299-be99-8f859cf6c7bb service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:34.995290 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1089/2192] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:09:34 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 124 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:09:38.726964 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae9d9d-0acc-4cc7-9861-8938d53d8362 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:38.814997 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:38.844443 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ae9d9d-0acc-4cc7-9861-8938d53d8362 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:38.854427 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e66ef-3f0a-4099-99c5-ee538e0a7a0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:38.855952 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e66ef-3f0a-4099-99c5-ee538e0a7a0b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:38.857217 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e66ef-3f0a-4099-99c5-ee538e0a7a0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:38.858051 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea0e66ef-3f0a-4099-99c5-ee538e0a7a0b admin admin] PATH_INFO: `/v3/users/fc671d6fc7a645ce9f6356f30e318f7a` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:38.861203 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e66ef-3f0a-4099-99c5-ee538e0a7a0b admin admin] RBAC: Authorizing `identity:delete_user(user_id=fc671d6fc7a645ce9f6356f30e318f7a)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:38.865973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea0e66ef-3f0a-4099-99c5-ee538e0a7a0b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:38.949968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ea0e66ef-3f0a-4099-99c5-ee538e0a7a0b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fc671d6fc7a645ce9f6356f30e318f7a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:38.974243 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ea0e66ef-3f0a-4099-99c5-ee538e0a7a0b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fc671d6fc7a645ce9f6356f30e318f7a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:38.990770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ea0e66ef-3f0a-4099-99c5-ee538e0a7a0b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fc671d6fc7a645ce9f6356f30e318f7a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:38.992104 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ea0e66ef-3f0a-4099-99c5-ee538e0a7a0b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fc671d6fc7a645ce9f6356f30e318f7a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:39.004018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ea0e66ef-3f0a-4099-99c5-ee538e0a7a0b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fc671d6fc7a645ce9f6356f30e318f7a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:39.011056 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1104/2193] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:09:38 2022] DELETE /identity/v3/users/fc671d6fc7a645ce9f6356f30e318f7a => generated 0 bytes in 286 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:39.016058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3eee69-0023-41d6-ae01-2268ebdb4ae1 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:39.017055 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3eee69-0023-41d6-ae01-2268ebdb4ae1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:39.017350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3eee69-0023-41d6-ae01-2268ebdb4ae1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:39.038551 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cc3eee69-0023-41d6-ae01-2268ebdb4ae1 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:39.039788 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cc3eee69-0023-41d6-ae01-2268ebdb4ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:39.114568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:39.124669 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1090/2194] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:39.135673 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e66ef-3f0a-4099-99c5-ee538e0a7a0b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:39.137222 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ea0e66ef-3f0a-4099-99c5-ee538e0a7a0b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:39.208192 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea0e66ef-3f0a-4099-99c5-ee538e0a7a0b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:39.211968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b93e57-ba6b-4b98-8b1f-5f79efd1b2d8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:39.212641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b93e57-ba6b-4b98-8b1f-5f79efd1b2d8 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:39.212929 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b93e57-ba6b-4b98-8b1f-5f79efd1b2d8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:39.213208 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b93e57-ba6b-4b98-8b1f-5f79efd1b2d8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:39.224073 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b93e57-ba6b-4b98-8b1f-5f79efd1b2d8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:39.225610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b93e57-ba6b-4b98-8b1f-5f79efd1b2d8 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:39.237344 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1105/2195] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:09:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:09:39.572029 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3eee69-0023-41d6-ae01-2268ebdb4ae1 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:39.573494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cc3eee69-0023-41d6-ae01-2268ebdb4ae1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:39.587027 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3eee69-0023-41d6-ae01-2268ebdb4ae1 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:39.613326 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8b37e-fb01-4a1d-b2a2-f8b3d2ba7e19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:39.614233 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8b37e-fb01-4a1d-b2a2-f8b3d2ba7e19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:39.614558 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8b37e-fb01-4a1d-b2a2-f8b3d2ba7e19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:39.614829 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b8b37e-fb01-4a1d-b2a2-f8b3d2ba7e19 admin admin] PATH_INFO: `/v3/projects/e94edc8de70a4c5d8197b6cdfd3238f3` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:39.619286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8b37e-fb01-4a1d-b2a2-f8b3d2ba7e19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e94edc8de70a4c5d8197b6cdfd3238f3)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:39.621493 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b8b37e-fb01-4a1d-b2a2-f8b3d2ba7e19 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:39.695714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a1b8b37e-fb01-4a1d-b2a2-f8b3d2ba7e19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e94edc8de70a4c5d8197b6cdfd3238f3', 'request_id': 'req-a1b8b37e-fb01-4a1d-b2a2-f8b3d2ba7e19'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:39.704855 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a1b8b37e-fb01-4a1d-b2a2-f8b3d2ba7e19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e94edc8de70a4c5d8197b6cdfd3238f3', 'request_id': 'req-a1b8b37e-fb01-4a1d-b2a2-f8b3d2ba7e19'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:39.725238 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a1b8b37e-fb01-4a1d-b2a2-f8b3d2ba7e19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e94edc8de70a4c5d8197b6cdfd3238f3', 'request_id': 'req-a1b8b37e-fb01-4a1d-b2a2-f8b3d2ba7e19'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:39.740873 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a1b8b37e-fb01-4a1d-b2a2-f8b3d2ba7e19 admin admin] The token cache is being invalidate because project e94edc8de70a4c5d8197b6cdfd3238f3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:09:39.741143 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a1b8b37e-fb01-4a1d-b2a2-f8b3d2ba7e19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:39.741833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a1b8b37e-fb01-4a1d-b2a2-f8b3d2ba7e19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:39.743759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1091/2196] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:09:39 2022] DELETE /identity/v3/projects/e94edc8de70a4c5d8197b6cdfd3238f3 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:39.748509 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b93e57-ba6b-4b98-8b1f-5f79efd1b2d8 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:39.770025 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b93e57-ba6b-4b98-8b1f-5f79efd1b2d8 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:39.864964 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3313d193-22cc-4130-80c6-ed93ea6f7cf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:39.866506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313d193-22cc-4130-80c6-ed93ea6f7cf2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:39.870782 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313d193-22cc-4130-80c6-ed93ea6f7cf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:39.871324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3313d193-22cc-4130-80c6-ed93ea6f7cf2 admin admin] PATH_INFO: `/v3/users/27770228305b454fa004d5e78883ffd7` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:39.873043 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313d193-22cc-4130-80c6-ed93ea6f7cf2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=27770228305b454fa004d5e78883ffd7)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:39.875678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3313d193-22cc-4130-80c6-ed93ea6f7cf2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:39.895442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:39.963798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3313d193-22cc-4130-80c6-ed93ea6f7cf2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '27770228305b454fa004d5e78883ffd7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:39.994514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3313d193-22cc-4130-80c6-ed93ea6f7cf2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '27770228305b454fa004d5e78883ffd7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:40.006275 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3313d193-22cc-4130-80c6-ed93ea6f7cf2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '27770228305b454fa004d5e78883ffd7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:40.007231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3313d193-22cc-4130-80c6-ed93ea6f7cf2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '27770228305b454fa004d5e78883ffd7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:40.019254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-3313d193-22cc-4130-80c6-ed93ea6f7cf2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '27770228305b454fa004d5e78883ffd7'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:40.035176 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1106/2197] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:09:39 2022] DELETE /identity/v3/users/27770228305b454fa004d5e78883ffd7 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:09:40.425641 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8b37e-fb01-4a1d-b2a2-f8b3d2ba7e19 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:40.427938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a1b8b37e-fb01-4a1d-b2a2-f8b3d2ba7e19 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:40.441687 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b8b37e-fb01-4a1d-b2a2-f8b3d2ba7e19 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:40.512536 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb88602-8687-47b0-82eb-a36331f34296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:40.513442 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb88602-8687-47b0-82eb-a36331f34296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:40.513725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb88602-8687-47b0-82eb-a36331f34296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:40.514009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb88602-8687-47b0-82eb-a36331f34296 admin admin] PATH_INFO: `/v3/projects/2784d06d34e448edb81bacb230fbed55` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:40.515323 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb88602-8687-47b0-82eb-a36331f34296 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2784d06d34e448edb81bacb230fbed55)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:40.517271 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb88602-8687-47b0-82eb-a36331f34296 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:40.571673 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fcb88602-8687-47b0-82eb-a36331f34296 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2784d06d34e448edb81bacb230fbed55', 'request_id': 'req-fcb88602-8687-47b0-82eb-a36331f34296'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:40.578399 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fcb88602-8687-47b0-82eb-a36331f34296 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2784d06d34e448edb81bacb230fbed55', 'request_id': 'req-fcb88602-8687-47b0-82eb-a36331f34296'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:40.588199 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fcb88602-8687-47b0-82eb-a36331f34296 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2784d06d34e448edb81bacb230fbed55', 'request_id': 'req-fcb88602-8687-47b0-82eb-a36331f34296'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:40.601549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fcb88602-8687-47b0-82eb-a36331f34296 admin admin] The token cache is being invalidate because project 2784d06d34e448edb81bacb230fbed55 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:09:40.601853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fcb88602-8687-47b0-82eb-a36331f34296 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:40.602563 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fcb88602-8687-47b0-82eb-a36331f34296 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:40.604175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1092/2198] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:09:40 2022] DELETE /identity/v3/projects/2784d06d34e448edb81bacb230fbed55 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:40.609358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872be75-7237-49c2-8c59-30013288e0be None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:40.609688 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872be75-7237-49c2-8c59-30013288e0be None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:40.610032 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c872be75-7237-49c2-8c59-30013288e0be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:40.630024 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c872be75-7237-49c2-8c59-30013288e0be None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:40.631319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c872be75-7237-49c2-8c59-30013288e0be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:40.704998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1107/2199] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:40.712992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a722b-5eed-4f53-92e4-8de0b5a04ed7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:40.713305 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a722b-5eed-4f53-92e4-8de0b5a04ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:40.713544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42a722b-5eed-4f53-92e4-8de0b5a04ed7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:40.733666 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a42a722b-5eed-4f53-92e4-8de0b5a04ed7 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:40.734925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a42a722b-5eed-4f53-92e4-8de0b5a04ed7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:40.746191 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1093/2200] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:40.750621 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872be75-7237-49c2-8c59-30013288e0be None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:40.761121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c872be75-7237-49c2-8c59-30013288e0be None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:40.765933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720fd4a-25a6-486f-b4d6-0c3507485009 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:40.766653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720fd4a-25a6-486f-b4d6-0c3507485009 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:40.766938 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720fd4a-25a6-486f-b4d6-0c3507485009 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:40.767212 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3720fd4a-25a6-486f-b4d6-0c3507485009 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:40.767957 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3720fd4a-25a6-486f-b4d6-0c3507485009 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:09:40.770373 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3720fd4a-25a6-486f-b4d6-0c3507485009 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:40.770584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3720fd4a-25a6-486f-b4d6-0c3507485009 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:40.784662 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1108/2201] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:09:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:09:40.793036 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a722b-5eed-4f53-92e4-8de0b5a04ed7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:40.805403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:40.812807 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42a722b-5eed-4f53-92e4-8de0b5a04ed7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:40.823568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733488bf-3e80-4a4c-b94e-bc4fed881a92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:40.824738 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733488bf-3e80-4a4c-b94e-bc4fed881a92 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:40.825306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733488bf-3e80-4a4c-b94e-bc4fed881a92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:40.825872 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733488bf-3e80-4a4c-b94e-bc4fed881a92 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:40.827224 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733488bf-3e80-4a4c-b94e-bc4fed881a92 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:40.831420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733488bf-3e80-4a4c-b94e-bc4fed881a92 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:40.875091 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1094/2202] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:09:40 2022] POST /identity/v3/projects => generated 401 bytes in 83 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:09:40.880894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720fd4a-25a6-486f-b4d6-0c3507485009 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:40.894798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3720fd4a-25a6-486f-b4d6-0c3507485009 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:40.993475 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb0977-48b5-4e03-a2f2-de240d81fed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:40.994293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb0977-48b5-4e03-a2f2-de240d81fed3 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:40.994935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb0977-48b5-4e03-a2f2-de240d81fed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:40.995155 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdb0977-48b5-4e03-a2f2-de240d81fed3 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:40.995739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb0977-48b5-4e03-a2f2-de240d81fed3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:40.997818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdb0977-48b5-4e03-a2f2-de240d81fed3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:41.021365 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1109/2203] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:09:40 2022] POST /identity/v3/users => generated 301 bytes in 141 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:09:41.028802 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733488bf-3e80-4a4c-b94e-bc4fed881a92 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:41.041460 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733488bf-3e80-4a4c-b94e-bc4fed881a92 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:41.057473 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b5406-7cc1-4a65-88cf-d72353ff05f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:41.063694 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b5406-7cc1-4a65-88cf-d72353ff05f8 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:41.064313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b5406-7cc1-4a65-88cf-d72353ff05f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:41.064642 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de6b5406-7cc1-4a65-88cf-d72353ff05f8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:41.065494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b5406-7cc1-4a65-88cf-d72353ff05f8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:41.067168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de6b5406-7cc1-4a65-88cf-d72353ff05f8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:41.085269 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1095/2204] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:09:41.091713 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb0977-48b5-4e03-a2f2-de240d81fed3 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:41.099090 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:41.103399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdb0977-48b5-4e03-a2f2-de240d81fed3 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:41.111104 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a78bf7-a40d-4376-a35f-4ea9b476a43f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:41.113021 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a78bf7-a40d-4376-a35f-4ea9b476a43f admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:41.113595 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a78bf7-a40d-4376-a35f-4ea9b476a43f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:41.114008 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a78bf7-a40d-4376-a35f-4ea9b476a43f admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:41.115081 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a78bf7-a40d-4376-a35f-4ea9b476a43f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:41.116838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a78bf7-a40d-4376-a35f-4ea9b476a43f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:41.128604 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1110/2205] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:09:41.133736 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b5406-7cc1-4a65-88cf-d72353ff05f8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:41.145252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de6b5406-7cc1-4a65-88cf-d72353ff05f8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:41.161520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff242baa-7c82-4196-ac55-8f78a0c5609a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:41.163306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff242baa-7c82-4196-ac55-8f78a0c5609a admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:41.163904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff242baa-7c82-4196-ac55-8f78a0c5609a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:41.164494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff242baa-7c82-4196-ac55-8f78a0c5609a admin admin] PATH_INFO: `/v3/projects/c8e8c64e7ea948dabb3c79292fe8fba3/users/2c77ee87fc2d42d8bec6771e8614de52/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:41.193752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff242baa-7c82-4196-ac55-8f78a0c5609a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c8e8c64e7ea948dabb3c79292fe8fba3, user_id=2c77ee87fc2d42d8bec6771e8614de52, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:41.199225 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff242baa-7c82-4196-ac55-8f78a0c5609a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:41.218934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1096/2206] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:09:41 2022] PUT /identity/v3/projects/c8e8c64e7ea948dabb3c79292fe8fba3/users/2c77ee87fc2d42d8bec6771e8614de52/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:09:41.236515 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701d636-71fa-440e-a4e8-e66ce1cc9147 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:41.236972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701d636-71fa-440e-a4e8-e66ce1cc9147 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:41.237287 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4701d636-71fa-440e-a4e8-e66ce1cc9147 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:41.280345 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4701d636-71fa-440e-a4e8-e66ce1cc9147 None None] MFA Rules not processed for user `2c77ee87fc2d42d8bec6771e8614de52`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:41.281740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4701d636-71fa-440e-a4e8-e66ce1cc9147 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:41.373599 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1111/2207] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:41 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:41.378679 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff242baa-7c82-4196-ac55-8f78a0c5609a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:41.461239 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff242baa-7c82-4196-ac55-8f78a0c5609a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:41.467639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f11d3-8b4c-4f1e-ab49-ba64a8caaeb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:41.468338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f11d3-8b4c-4f1e-ab49-ba64a8caaeb5 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:41.468636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f11d3-8b4c-4f1e-ab49-ba64a8caaeb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:41.468918 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-233f11d3-8b4c-4f1e-ab49-ba64a8caaeb5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:41.469674 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f11d3-8b4c-4f1e-ab49-ba64a8caaeb5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:41.471800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-233f11d3-8b4c-4f1e-ab49-ba64a8caaeb5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:41.505262 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1097/2208] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:09:41 2022] POST /identity/v3/projects => generated 401 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:09:41.510372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701d636-71fa-440e-a4e8-e66ce1cc9147 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:41.522543 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4701d636-71fa-440e-a4e8-e66ce1cc9147 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:41.598261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e07d0-ba60-4915-9e08-cbd044c20834 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:41.598898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e07d0-ba60-4915-9e08-cbd044c20834 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:41.599209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e07d0-ba60-4915-9e08-cbd044c20834 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:41.599495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f1e07d0-ba60-4915-9e08-cbd044c20834 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:41.600214 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e07d0-ba60-4915-9e08-cbd044c20834 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:41.602173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f1e07d0-ba60-4915-9e08-cbd044c20834 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:41.624772 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1112/2209] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:09:41 2022] POST /identity/v3/users => generated 307 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:09:41.629825 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f11d3-8b4c-4f1e-ab49-ba64a8caaeb5 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:41.640899 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-233f11d3-8b4c-4f1e-ab49-ba64a8caaeb5 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:41.646464 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43010a9-9000-48b8-91e6-d8a94dc5b851 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:41.647155 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43010a9-9000-48b8-91e6-d8a94dc5b851 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:41.647552 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43010a9-9000-48b8-91e6-d8a94dc5b851 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:41.647835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f43010a9-9000-48b8-91e6-d8a94dc5b851 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:41.648641 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43010a9-9000-48b8-91e6-d8a94dc5b851 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:41.650349 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f43010a9-9000-48b8-91e6-d8a94dc5b851 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:41.663857 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1098/2210] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:09:41.667893 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e07d0-ba60-4915-9e08-cbd044c20834 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:41.680591 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f1e07d0-ba60-4915-9e08-cbd044c20834 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:41.683785 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b450a4c-15aa-4363-bd8f-49bc08c7f5f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:41.684689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b450a4c-15aa-4363-bd8f-49bc08c7f5f2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:41.684970 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b450a4c-15aa-4363-bd8f-49bc08c7f5f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:41.685245 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b450a4c-15aa-4363-bd8f-49bc08c7f5f2 admin admin] PATH_INFO: `/v3/projects/afcf9911828244af903d321e5ee95341/users/4acdf2d766f04812bc06b191efb3d766/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:41.703519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b450a4c-15aa-4363-bd8f-49bc08c7f5f2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afcf9911828244af903d321e5ee95341, user_id=4acdf2d766f04812bc06b191efb3d766, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:41.706869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b450a4c-15aa-4363-bd8f-49bc08c7f5f2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:41.723935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1113/2211] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:09:41 2022] PUT /identity/v3/projects/afcf9911828244af903d321e5ee95341/users/4acdf2d766f04812bc06b191efb3d766/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:41.728019 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43010a9-9000-48b8-91e6-d8a94dc5b851 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:41.739639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f43010a9-9000-48b8-91e6-d8a94dc5b851 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:41.802504 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8ddedd-a054-408d-a344-14668d0f321f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:41.803182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ddedd-a054-408d-a344-14668d0f321f admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:41.803463 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ddedd-a054-408d-a344-14668d0f321f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:41.803739 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8ddedd-a054-408d-a344-14668d0f321f admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:41.804482 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ddedd-a054-408d-a344-14668d0f321f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:41.806078 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8ddedd-a054-408d-a344-14668d0f321f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:41.817380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1099/2212] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:41 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:09:41.821513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b450a4c-15aa-4363-bd8f-49bc08c7f5f2 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:41.832883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b450a4c-15aa-4363-bd8f-49bc08c7f5f2 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:41.837994 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02777c24-88f4-4865-97a7-2682a30021a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:41.838944 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02777c24-88f4-4865-97a7-2682a30021a2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:41.839233 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02777c24-88f4-4865-97a7-2682a30021a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:41.839512 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02777c24-88f4-4865-97a7-2682a30021a2 admin admin] PATH_INFO: `/v3/projects/afcf9911828244af903d321e5ee95341/users/4acdf2d766f04812bc06b191efb3d766/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:41.841574 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02777c24-88f4-4865-97a7-2682a30021a2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=afcf9911828244af903d321e5ee95341, user_id=4acdf2d766f04812bc06b191efb3d766, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:41.844333 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02777c24-88f4-4865-97a7-2682a30021a2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:41.855422 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1114/2213] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:09:41 2022] PUT /identity/v3/projects/afcf9911828244af903d321e5ee95341/users/4acdf2d766f04812bc06b191efb3d766/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 34 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:41.868234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2af36-b8a7-4995-afe4-f728cfbfb476 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:41.868632 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2af36-b8a7-4995-afe4-f728cfbfb476 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:41.869207 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea2af36-b8a7-4995-afe4-f728cfbfb476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:41.899633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cea2af36-b8a7-4995-afe4-f728cfbfb476 None None] MFA Rules not processed for user `4acdf2d766f04812bc06b191efb3d766`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:41.900905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cea2af36-b8a7-4995-afe4-f728cfbfb476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:41.906711 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:41.978231 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1100/2214] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:41 2022] POST /identity/v3/auth/tokens => generated 3335 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:41.989437 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02777c24-88f4-4865-97a7-2682a30021a2 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:41.991316 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-02777c24-88f4-4865-97a7-2682a30021a2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:42.035649 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:42.062838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02777c24-88f4-4865-97a7-2682a30021a2 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:42.067235 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ee227-df0a-4e22-819d-dc7a2d45fdad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:42.067916 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ee227-df0a-4e22-819d-dc7a2d45fdad service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:42.068205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ee227-df0a-4e22-819d-dc7a2d45fdad service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:42.068488 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb8ee227-df0a-4e22-819d-dc7a2d45fdad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:42.085259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ee227-df0a-4e22-819d-dc7a2d45fdad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:42.095410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb8ee227-df0a-4e22-819d-dc7a2d45fdad service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:42.100690 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1115/2215] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:09:41 2022] GET /identity/v3/auth/tokens => generated 3335 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:09:42.188160 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2af36-b8a7-4995-afe4-f728cfbfb476 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:42.198732 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea2af36-b8a7-4995-afe4-f728cfbfb476 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:42.202750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f90ce4-87f1-4005-aa1f-9d0b5c8f4438 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:42.203517 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f90ce4-87f1-4005-aa1f-9d0b5c8f4438 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:42.203946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f90ce4-87f1-4005-aa1f-9d0b5c8f4438 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:42.204234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f90ce4-87f1-4005-aa1f-9d0b5c8f4438 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:42.269708 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f90ce4-87f1-4005-aa1f-9d0b5c8f4438 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:42.271202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f90ce4-87f1-4005-aa1f-9d0b5c8f4438 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:42.282188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1101/2216] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:09:42 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:09:47.754614 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ee227-df0a-4e22-819d-dc7a2d45fdad service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:47.850147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb8ee227-df0a-4e22-819d-dc7a2d45fdad service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:47.855283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4ba47-1301-4620-9115-667735562f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:47.856675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4ba47-1301-4620-9115-667735562f20 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:47.857017 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4ba47-1301-4620-9115-667735562f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:47.857498 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ba4ba47-1301-4620-9115-667735562f20 admin admin] PATH_INFO: `/v3/users/2c77ee87fc2d42d8bec6771e8614de52` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:47.858851 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4ba47-1301-4620-9115-667735562f20 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2c77ee87fc2d42d8bec6771e8614de52)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:47.861057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ba4ba47-1301-4620-9115-667735562f20 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:47.933561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8ba4ba47-1301-4620-9115-667735562f20 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2c77ee87fc2d42d8bec6771e8614de52'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:47.954974 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8ba4ba47-1301-4620-9115-667735562f20 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2c77ee87fc2d42d8bec6771e8614de52'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:47.969096 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8ba4ba47-1301-4620-9115-667735562f20 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2c77ee87fc2d42d8bec6771e8614de52'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:47.969875 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8ba4ba47-1301-4620-9115-667735562f20 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2c77ee87fc2d42d8bec6771e8614de52'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:47.979498 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8ba4ba47-1301-4620-9115-667735562f20 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2c77ee87fc2d42d8bec6771e8614de52'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:47.987938 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1116/2217] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:09:47 2022] DELETE /identity/v3/users/2c77ee87fc2d42d8bec6771e8614de52 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:47.992181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5967f3-328f-49d7-8269-a4e619960686 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:47.992732 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5967f3-328f-49d7-8269-a4e619960686 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:47.997814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5967f3-328f-49d7-8269-a4e619960686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:48.001688 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4ba47-1301-4620-9115-667735562f20 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:48.004595 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8ba4ba47-1301-4620-9115-667735562f20 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:48.021031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ce5967f3-328f-49d7-8269-a4e619960686 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:48.023168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ce5967f3-328f-49d7-8269-a4e619960686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:48.093430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ba4ba47-1301-4620-9115-667735562f20 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:48.097500 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883400c-ea55-474d-aebe-6bbdb9cd305b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:48.098361 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883400c-ea55-474d-aebe-6bbdb9cd305b service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:48.098658 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883400c-ea55-474d-aebe-6bbdb9cd305b service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:48.098938 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e883400c-ea55-474d-aebe-6bbdb9cd305b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:48.100812 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e883400c-ea55-474d-aebe-6bbdb9cd305b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:48.102773 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1102/2218] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:09:48.112970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5967f3-328f-49d7-8269-a4e619960686 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:48.116020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:48.129516 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5967f3-328f-49d7-8269-a4e619960686 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:48.138835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace35143-3c9e-404e-97de-79c1baebd353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:48.139621 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace35143-3c9e-404e-97de-79c1baebd353 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:48.139981 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace35143-3c9e-404e-97de-79c1baebd353 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:48.140332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace35143-3c9e-404e-97de-79c1baebd353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:48.153055 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace35143-3c9e-404e-97de-79c1baebd353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:48.155146 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ace35143-3c9e-404e-97de-79c1baebd353 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:48.169551 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1103/2219] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:09:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:09:48.188431 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883400c-ea55-474d-aebe-6bbdb9cd305b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:48.190356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e883400c-ea55-474d-aebe-6bbdb9cd305b service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:48.201444 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1117/2220] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:09:48 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:09:48.931314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace35143-3c9e-404e-97de-79c1baebd353 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:48.933627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ace35143-3c9e-404e-97de-79c1baebd353 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:48.968260 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace35143-3c9e-404e-97de-79c1baebd353 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:48.982815 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12151fdd-410c-4818-9724-d01d19ebb729 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:48.984361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12151fdd-410c-4818-9724-d01d19ebb729 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:48.984969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12151fdd-410c-4818-9724-d01d19ebb729 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:48.985617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12151fdd-410c-4818-9724-d01d19ebb729 admin admin] PATH_INFO: `/v3/projects/c8e8c64e7ea948dabb3c79292fe8fba3` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:48.990263 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12151fdd-410c-4818-9724-d01d19ebb729 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c8e8c64e7ea948dabb3c79292fe8fba3)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:48.994544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12151fdd-410c-4818-9724-d01d19ebb729 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:49.042378 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:49.088356 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-12151fdd-410c-4818-9724-d01d19ebb729 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c8e8c64e7ea948dabb3c79292fe8fba3', 'request_id': 'req-12151fdd-410c-4818-9724-d01d19ebb729'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:49.097055 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-12151fdd-410c-4818-9724-d01d19ebb729 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c8e8c64e7ea948dabb3c79292fe8fba3', 'request_id': 'req-12151fdd-410c-4818-9724-d01d19ebb729'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:49.108150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-12151fdd-410c-4818-9724-d01d19ebb729 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c8e8c64e7ea948dabb3c79292fe8fba3', 'request_id': 'req-12151fdd-410c-4818-9724-d01d19ebb729'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:49.125557 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-12151fdd-410c-4818-9724-d01d19ebb729 admin admin] The token cache is being invalidate because project c8e8c64e7ea948dabb3c79292fe8fba3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:09:49.125557 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-12151fdd-410c-4818-9724-d01d19ebb729 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:49.126271 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-12151fdd-410c-4818-9724-d01d19ebb729 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:49.131907 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1104/2221] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:09:48 2022] DELETE /identity/v3/projects/c8e8c64e7ea948dabb3c79292fe8fba3 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:49.133166 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883400c-ea55-474d-aebe-6bbdb9cd305b service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:49.145299 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e883400c-ea55-474d-aebe-6bbdb9cd305b service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:49.217534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffb7916-23b9-4f46-a9f2-b353f83261b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:49.218460 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb7916-23b9-4f46-a9f2-b353f83261b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:49.218744 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb7916-23b9-4f46-a9f2-b353f83261b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:49.219108 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffb7916-23b9-4f46-a9f2-b353f83261b8 admin admin] PATH_INFO: `/v3/users/4acdf2d766f04812bc06b191efb3d766` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:49.222325 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb7916-23b9-4f46-a9f2-b353f83261b8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4acdf2d766f04812bc06b191efb3d766)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:49.223182 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffb7916-23b9-4f46-a9f2-b353f83261b8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:49.271889 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1ffb7916-23b9-4f46-a9f2-b353f83261b8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4acdf2d766f04812bc06b191efb3d766'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:49.286215 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1ffb7916-23b9-4f46-a9f2-b353f83261b8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4acdf2d766f04812bc06b191efb3d766'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:49.294750 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1ffb7916-23b9-4f46-a9f2-b353f83261b8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4acdf2d766f04812bc06b191efb3d766'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:49.295471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1ffb7916-23b9-4f46-a9f2-b353f83261b8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4acdf2d766f04812bc06b191efb3d766'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:49.303800 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1ffb7916-23b9-4f46-a9f2-b353f83261b8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4acdf2d766f04812bc06b191efb3d766'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:49.310091 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1118/2222] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:09:49 2022] DELETE /identity/v3/users/4acdf2d766f04812bc06b191efb3d766 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:49.609753 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12151fdd-410c-4818-9724-d01d19ebb729 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:49.611718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-12151fdd-410c-4818-9724-d01d19ebb729 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:49.626251 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12151fdd-410c-4818-9724-d01d19ebb729 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:49.697425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61dbcd3e-a111-446e-8cc3-f3c918bf98ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:49.698389 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbcd3e-a111-446e-8cc3-f3c918bf98ca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:49.698780 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbcd3e-a111-446e-8cc3-f3c918bf98ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:49.699065 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61dbcd3e-a111-446e-8cc3-f3c918bf98ca admin admin] PATH_INFO: `/v3/projects/afcf9911828244af903d321e5ee95341` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:49.700295 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dbcd3e-a111-446e-8cc3-f3c918bf98ca admin admin] RBAC: Authorizing `identity:delete_project(project_id=afcf9911828244af903d321e5ee95341)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:49.702906 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61dbcd3e-a111-446e-8cc3-f3c918bf98ca admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:49.751417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-61dbcd3e-a111-446e-8cc3-f3c918bf98ca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'afcf9911828244af903d321e5ee95341', 'request_id': 'req-61dbcd3e-a111-446e-8cc3-f3c918bf98ca'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:49.757682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-61dbcd3e-a111-446e-8cc3-f3c918bf98ca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'afcf9911828244af903d321e5ee95341', 'request_id': 'req-61dbcd3e-a111-446e-8cc3-f3c918bf98ca'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:49.765380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-61dbcd3e-a111-446e-8cc3-f3c918bf98ca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'afcf9911828244af903d321e5ee95341', 'request_id': 'req-61dbcd3e-a111-446e-8cc3-f3c918bf98ca'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:49.778225 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-61dbcd3e-a111-446e-8cc3-f3c918bf98ca admin admin] The token cache is being invalidate because project afcf9911828244af903d321e5ee95341 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:09:49.778539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-61dbcd3e-a111-446e-8cc3-f3c918bf98ca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:49.779192 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-61dbcd3e-a111-446e-8cc3-f3c918bf98ca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:49.780973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1105/2223] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:09:49 2022] DELETE /identity/v3/projects/afcf9911828244af903d321e5ee95341 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:49.785941 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356ce3e-4be3-4703-92de-4eef0066ca2e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:49.786324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356ce3e-4be3-4703-92de-4eef0066ca2e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:49.786694 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3356ce3e-4be3-4703-92de-4eef0066ca2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:49.805887 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-3356ce3e-4be3-4703-92de-4eef0066ca2e None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:49.807134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3356ce3e-4be3-4703-92de-4eef0066ca2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:49.830089 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:49.882331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1119/2224] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:49.890349 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc748f-f7fe-421d-b780-aaeda766071f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:49.894971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc748f-f7fe-421d-b780-aaeda766071f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:49.896719 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31bc748f-f7fe-421d-b780-aaeda766071f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:49.922880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-31bc748f-f7fe-421d-b780-aaeda766071f None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:49.923436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-31bc748f-f7fe-421d-b780-aaeda766071f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:49.934047 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1106/2225] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:49.938500 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356ce3e-4be3-4703-92de-4eef0066ca2e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:49.959047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3356ce3e-4be3-4703-92de-4eef0066ca2e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:49.963811 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84817706-5435-4990-8746-4bf522900c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:49.964484 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84817706-5435-4990-8746-4bf522900c85 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:49.964770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84817706-5435-4990-8746-4bf522900c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:49.965200 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84817706-5435-4990-8746-4bf522900c85 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:49.965945 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84817706-5435-4990-8746-4bf522900c85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:09:49.966258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84817706-5435-4990-8746-4bf522900c85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:49.967985 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84817706-5435-4990-8746-4bf522900c85 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:49.980684 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1120/2226] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:09:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:09:49.985115 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc748f-f7fe-421d-b780-aaeda766071f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:49.996488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31bc748f-f7fe-421d-b780-aaeda766071f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:50.001645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0cca08-a0ff-4298-90ca-d9da0147bcf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:50.002463 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cca08-a0ff-4298-90ca-d9da0147bcf9 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:50.002819 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cca08-a0ff-4298-90ca-d9da0147bcf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:50.003165 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0cca08-a0ff-4298-90ca-d9da0147bcf9 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:50.003955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0cca08-a0ff-4298-90ca-d9da0147bcf9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:50.005958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0cca08-a0ff-4298-90ca-d9da0147bcf9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:50.034592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1107/2227] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:09:49 2022] POST /identity/v3/projects => generated 377 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:09:50.039145 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84817706-5435-4990-8746-4bf522900c85 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:50.054970 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84817706-5435-4990-8746-4bf522900c85 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:50.118883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853db35-57be-458c-a2c6-ad2bdb9333a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:50.119697 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853db35-57be-458c-a2c6-ad2bdb9333a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:50.119987 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853db35-57be-458c-a2c6-ad2bdb9333a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:50.120302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3853db35-57be-458c-a2c6-ad2bdb9333a7 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:50.121072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853db35-57be-458c-a2c6-ad2bdb9333a7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:50.123051 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3853db35-57be-458c-a2c6-ad2bdb9333a7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:50.149266 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1121/2228] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:09:50 2022] POST /identity/v3/users => generated 289 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:09:50.150536 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0cca08-a0ff-4298-90ca-d9da0147bcf9 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:50.161119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0cca08-a0ff-4298-90ca-d9da0147bcf9 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:50.165804 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3517a85-3d26-4639-a2c8-941b20d69498 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:50.166506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3517a85-3d26-4639-a2c8-941b20d69498 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:50.166789 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3517a85-3d26-4639-a2c8-941b20d69498 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:50.167063 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3517a85-3d26-4639-a2c8-941b20d69498 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:50.167834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3517a85-3d26-4639-a2c8-941b20d69498 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:50.169422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3517a85-3d26-4639-a2c8-941b20d69498 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:50.182174 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1108/2229] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:09:50.186793 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853db35-57be-458c-a2c6-ad2bdb9333a7 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:50.197263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3853db35-57be-458c-a2c6-ad2bdb9333a7 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:50.202043 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2fd443f-411b-4c9b-8da8-d333fec063e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:50.202739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd443f-411b-4c9b-8da8-d333fec063e4 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:50.203021 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd443f-411b-4c9b-8da8-d333fec063e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:50.203297 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2fd443f-411b-4c9b-8da8-d333fec063e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:50.204023 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fd443f-411b-4c9b-8da8-d333fec063e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:50.206205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2fd443f-411b-4c9b-8da8-d333fec063e4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:50.216370 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1122/2230] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:09:50.220478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3517a85-3d26-4639-a2c8-941b20d69498 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:50.225644 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=84,keystone:DELETE=13,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:50.232026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3517a85-3d26-4639-a2c8-941b20d69498 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:50.238984 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a289f8d-6d9e-422a-8d1a-737cd355c832 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:50.239883 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a289f8d-6d9e-422a-8d1a-737cd355c832 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:50.240167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a289f8d-6d9e-422a-8d1a-737cd355c832 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:50.240445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a289f8d-6d9e-422a-8d1a-737cd355c832 admin admin] PATH_INFO: `/v3/projects/a6dcd28d638144efa1a7e4955c8b4477/users/c5a8b03c39b746c5994437f5e449ef1c/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:50.255937 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a289f8d-6d9e-422a-8d1a-737cd355c832 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6dcd28d638144efa1a7e4955c8b4477, user_id=c5a8b03c39b746c5994437f5e449ef1c, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:50.258629 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a289f8d-6d9e-422a-8d1a-737cd355c832 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:50.272850 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1109/2231] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:09:50 2022] PUT /identity/v3/projects/a6dcd28d638144efa1a7e4955c8b4477/users/c5a8b03c39b746c5994437f5e449ef1c/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:09:50.284186 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e365b-2fa6-4834-9970-489517dbdee1 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:50.284578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e365b-2fa6-4834-9970-489517dbdee1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:50.284932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4e365b-2fa6-4834-9970-489517dbdee1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:50.313248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-be4e365b-2fa6-4834-9970-489517dbdee1 None None] MFA Rules not processed for user `c5a8b03c39b746c5994437f5e449ef1c`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:50.314520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-be4e365b-2fa6-4834-9970-489517dbdee1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:50.381767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1123/2232] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:09:50.386254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a289f8d-6d9e-422a-8d1a-737cd355c832 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:50.458792 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a289f8d-6d9e-422a-8d1a-737cd355c832 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:50.465884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea8998-4316-4cc7-8caa-05f8c1a2c595 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:50.466742 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea8998-4316-4cc7-8caa-05f8c1a2c595 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:50.467052 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea8998-4316-4cc7-8caa-05f8c1a2c595 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:50.467327 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffea8998-4316-4cc7-8caa-05f8c1a2c595 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:50.468044 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea8998-4316-4cc7-8caa-05f8c1a2c595 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:50.470095 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffea8998-4316-4cc7-8caa-05f8c1a2c595 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:50.498847 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1110/2233] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:09:50 2022] POST /identity/v3/projects => generated 377 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:09:50.504480 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e365b-2fa6-4834-9970-489517dbdee1 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:50.516397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4e365b-2fa6-4834-9970-489517dbdee1 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:50.536434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:50.581505 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099acd3-af92-4ea2-9754-c84d0c92fd3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:50.582279 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099acd3-af92-4ea2-9754-c84d0c92fd3e admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:50.582599 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099acd3-af92-4ea2-9754-c84d0c92fd3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:50.582882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b099acd3-af92-4ea2-9754-c84d0c92fd3e admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:50.583599 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099acd3-af92-4ea2-9754-c84d0c92fd3e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:50.585536 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b099acd3-af92-4ea2-9754-c84d0c92fd3e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:50.605511 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1124/2234] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:09:50 2022] POST /identity/v3/users => generated 289 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:09:50.609619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea8998-4316-4cc7-8caa-05f8c1a2c595 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:50.620221 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffea8998-4316-4cc7-8caa-05f8c1a2c595 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:50.625203 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42feae7-f595-4019-b4da-075e056f0840 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:50.625890 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42feae7-f595-4019-b4da-075e056f0840 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:50.626173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42feae7-f595-4019-b4da-075e056f0840 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:50.626513 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a42feae7-f595-4019-b4da-075e056f0840 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:50.627904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42feae7-f595-4019-b4da-075e056f0840 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:50.629576 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a42feae7-f595-4019-b4da-075e056f0840 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:50.641144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1111/2235] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:09:50.645852 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099acd3-af92-4ea2-9754-c84d0c92fd3e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:50.656910 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b099acd3-af92-4ea2-9754-c84d0c92fd3e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:50.663232 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b88646-cbfb-4b2f-8875-8b3708065e35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:50.663232 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b88646-cbfb-4b2f-8875-8b3708065e35 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:50.663232 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b88646-cbfb-4b2f-8875-8b3708065e35 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:50.663232 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b88646-cbfb-4b2f-8875-8b3708065e35 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:50.663936 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b88646-cbfb-4b2f-8875-8b3708065e35 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:50.665564 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b88646-cbfb-4b2f-8875-8b3708065e35 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:50.677793 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1125/2236] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:09:50.682252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42feae7-f595-4019-b4da-075e056f0840 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:50.693671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a42feae7-f595-4019-b4da-075e056f0840 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:50.701047 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184da5cb-efa2-4603-9816-00b19cff7c4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:50.701242 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184da5cb-efa2-4603-9816-00b19cff7c4a admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:50.701601 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184da5cb-efa2-4603-9816-00b19cff7c4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:50.701946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-184da5cb-efa2-4603-9816-00b19cff7c4a admin admin] PATH_INFO: `/v3/projects/17d542a5136048cf8fc6310f300193f0/users/736705b33f034d828ecce1d13037f4ce/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:50.718950 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184da5cb-efa2-4603-9816-00b19cff7c4a admin admin] RBAC: Authorizing `identity:create_grant(project_id=17d542a5136048cf8fc6310f300193f0, user_id=736705b33f034d828ecce1d13037f4ce, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:50.721653 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-184da5cb-efa2-4603-9816-00b19cff7c4a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:50.736912 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1112/2237] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:09:50 2022] PUT /identity/v3/projects/17d542a5136048cf8fc6310f300193f0/users/736705b33f034d828ecce1d13037f4ce/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:50.747488 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e7c82-d816-4133-b172-47023207c118 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:50.748033 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e7c82-d816-4133-b172-47023207c118 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:50.748487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b6e7c82-d816-4133-b172-47023207c118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:50.778518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8b6e7c82-d816-4133-b172-47023207c118 None None] MFA Rules not processed for user `736705b33f034d828ecce1d13037f4ce`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:50.779869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8b6e7c82-d816-4133-b172-47023207c118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:50.854002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1126/2238] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:50 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:50.858595 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184da5cb-efa2-4603-9816-00b19cff7c4a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:50.928797 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-184da5cb-efa2-4603-9816-00b19cff7c4a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:50.933665 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dd743-f92b-4873-9e17-6467710d6e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:50.934388 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dd743-f92b-4873-9e17-6467710d6e3b admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:50.934824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dd743-f92b-4873-9e17-6467710d6e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:50.935119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562dd743-f92b-4873-9e17-6467710d6e3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:50.935982 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562dd743-f92b-4873-9e17-6467710d6e3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:50.937924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562dd743-f92b-4873-9e17-6467710d6e3b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:50.973592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1113/2239] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:09:50 2022] POST /identity/v3/projects => generated 379 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:09:50.977682 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e7c82-d816-4133-b172-47023207c118 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:50.994167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b6e7c82-d816-4133-b172-47023207c118 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:51.078524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792fbc6-6501-41af-8521-96eec28ecf3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:51.079247 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792fbc6-6501-41af-8521-96eec28ecf3f admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:51.079843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792fbc6-6501-41af-8521-96eec28ecf3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:51.080191 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2792fbc6-6501-41af-8521-96eec28ecf3f admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:51.080980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2792fbc6-6501-41af-8521-96eec28ecf3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:51.083000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2792fbc6-6501-41af-8521-96eec28ecf3f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:51.106125 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1127/2240] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:09:50 2022] POST /identity/v3/users => generated 296 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:09:51.110241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dd743-f92b-4873-9e17-6467710d6e3b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:51.121646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562dd743-f92b-4873-9e17-6467710d6e3b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:51.128053 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e8dc7-df5b-4ea6-b6bf-76dc37e86e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:51.128737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e8dc7-df5b-4ea6-b6bf-76dc37e86e2a admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:51.129026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e8dc7-df5b-4ea6-b6bf-76dc37e86e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:51.129308 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a75e8dc7-df5b-4ea6-b6bf-76dc37e86e2a admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:51.130110 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e8dc7-df5b-4ea6-b6bf-76dc37e86e2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:51.131819 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a75e8dc7-df5b-4ea6-b6bf-76dc37e86e2a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:51.143582 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1114/2241] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:09:51.147625 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792fbc6-6501-41af-8521-96eec28ecf3f admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:51.158468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2792fbc6-6501-41af-8521-96eec28ecf3f admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:51.163396 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de17fa-32cd-4053-9be5-f6b71050640c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:51.164377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de17fa-32cd-4053-9be5-f6b71050640c admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:51.164736 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de17fa-32cd-4053-9be5-f6b71050640c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:51.165095 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57de17fa-32cd-4053-9be5-f6b71050640c admin admin] PATH_INFO: `/v3/projects/c1155ee336024a50a941b1649f624a98/users/e427cea9ba584432a8cde6034e6f686f/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:51.182884 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de17fa-32cd-4053-9be5-f6b71050640c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1155ee336024a50a941b1649f624a98, user_id=e427cea9ba584432a8cde6034e6f686f, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:51.185810 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57de17fa-32cd-4053-9be5-f6b71050640c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:51.189667 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:51.204111 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1128/2242] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:09:51 2022] PUT /identity/v3/projects/c1155ee336024a50a941b1649f624a98/users/e427cea9ba584432a8cde6034e6f686f/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:09:51.208113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e8dc7-df5b-4ea6-b6bf-76dc37e86e2a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:51.213770 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:51.219455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a75e8dc7-df5b-4ea6-b6bf-76dc37e86e2a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:51.307836 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a929b7c-62f1-4ca8-9d4d-9691627fd0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:51.309150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a929b7c-62f1-4ca8-9d4d-9691627fd0b4 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:51.309150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a929b7c-62f1-4ca8-9d4d-9691627fd0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:51.309544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a929b7c-62f1-4ca8-9d4d-9691627fd0b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:51.311038 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a929b7c-62f1-4ca8-9d4d-9691627fd0b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:51.312301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a929b7c-62f1-4ca8-9d4d-9691627fd0b4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:51.323730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1115/2243] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:51 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:09:51.327880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de17fa-32cd-4053-9be5-f6b71050640c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:51.340650 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57de17fa-32cd-4053-9be5-f6b71050640c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:51.345707 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408612bd-8507-4881-9cd8-2e70ae865c70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:51.347195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408612bd-8507-4881-9cd8-2e70ae865c70 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:51.348026 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408612bd-8507-4881-9cd8-2e70ae865c70 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:51.348026 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408612bd-8507-4881-9cd8-2e70ae865c70 admin admin] PATH_INFO: `/v3/projects/c1155ee336024a50a941b1649f624a98/users/e427cea9ba584432a8cde6034e6f686f/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:51.349827 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408612bd-8507-4881-9cd8-2e70ae865c70 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1155ee336024a50a941b1649f624a98, user_id=e427cea9ba584432a8cde6034e6f686f, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:51.352790 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-408612bd-8507-4881-9cd8-2e70ae865c70 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:51.366689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1129/2244] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:09:51 2022] PUT /identity/v3/projects/c1155ee336024a50a941b1649f624a98/users/e427cea9ba584432a8cde6034e6f686f/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:51.376907 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9981c0-63eb-4980-8602-e35d6a875a97 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:51.377323 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9981c0-63eb-4980-8602-e35d6a875a97 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:51.377683 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9981c0-63eb-4980-8602-e35d6a875a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:51.417731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1c9981c0-63eb-4980-8602-e35d6a875a97 None None] MFA Rules not processed for user `e427cea9ba584432a8cde6034e6f686f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:51.419004 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1c9981c0-63eb-4980-8602-e35d6a875a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:51.519551 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1116/2245] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:51 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:09:51.520558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408612bd-8507-4881-9cd8-2e70ae865c70 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:51.522596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-408612bd-8507-4881-9cd8-2e70ae865c70 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:51.602522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408612bd-8507-4881-9cd8-2e70ae865c70 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:51.607534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb64de-d42b-4427-a814-cc28be214ac4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:51.608313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb64de-d42b-4427-a814-cc28be214ac4 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:51.608859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb64de-d42b-4427-a814-cc28be214ac4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:51.609147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4bb64de-d42b-4427-a814-cc28be214ac4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:51.620851 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb64de-d42b-4427-a814-cc28be214ac4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:51.622384 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4bb64de-d42b-4427-a814-cc28be214ac4 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:51.634660 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1130/2246] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:09:51 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:09:51.693178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9981c0-63eb-4980-8602-e35d6a875a97 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:51.694860 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1c9981c0-63eb-4980-8602-e35d6a875a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:51.710776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9981c0-63eb-4980-8602-e35d6a875a97 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:51.717741 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d15576a-47cd-4278-a044-2a28bb26f61e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:51.718728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15576a-47cd-4278-a044-2a28bb26f61e service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:51.719305 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15576a-47cd-4278-a044-2a28bb26f61e service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:51.719825 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d15576a-47cd-4278-a044-2a28bb26f61e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:51.792331 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15576a-47cd-4278-a044-2a28bb26f61e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:51.793839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d15576a-47cd-4278-a044-2a28bb26f61e service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:51.803919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1117/2247] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:09:51 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:09:54.730653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb64de-d42b-4427-a814-cc28be214ac4 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:54.796494 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4bb64de-d42b-4427-a814-cc28be214ac4 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:54.802083 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fbba71-28e3-4bf3-b7be-f67b8c12a2a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:54.803015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fbba71-28e3-4bf3-b7be-f67b8c12a2a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:54.803311 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fbba71-28e3-4bf3-b7be-f67b8c12a2a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:54.803599 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fbba71-28e3-4bf3-b7be-f67b8c12a2a1 admin admin] PATH_INFO: `/v3/users/c5a8b03c39b746c5994437f5e449ef1c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:54.804857 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fbba71-28e3-4bf3-b7be-f67b8c12a2a1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c5a8b03c39b746c5994437f5e449ef1c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:54.806936 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26fbba71-28e3-4bf3-b7be-f67b8c12a2a1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:54.862760 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-26fbba71-28e3-4bf3-b7be-f67b8c12a2a1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c5a8b03c39b746c5994437f5e449ef1c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:54.880645 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-26fbba71-28e3-4bf3-b7be-f67b8c12a2a1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c5a8b03c39b746c5994437f5e449ef1c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:54.889246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-26fbba71-28e3-4bf3-b7be-f67b8c12a2a1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c5a8b03c39b746c5994437f5e449ef1c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:54.890044 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-26fbba71-28e3-4bf3-b7be-f67b8c12a2a1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c5a8b03c39b746c5994437f5e449ef1c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:54.899084 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-26fbba71-28e3-4bf3-b7be-f67b8c12a2a1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c5a8b03c39b746c5994437f5e449ef1c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:54.906757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1131/2248] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:09:54 2022] DELETE /identity/v3/users/c5a8b03c39b746c5994437f5e449ef1c => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:09:54.912621 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41e523-9fce-41a0-9d00-c5bf950fd3de None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:54.913131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41e523-9fce-41a0-9d00-c5bf950fd3de None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:54.913233 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba41e523-9fce-41a0-9d00-c5bf950fd3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:54.937830 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ba41e523-9fce-41a0-9d00-c5bf950fd3de None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:54.939146 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ba41e523-9fce-41a0-9d00-c5bf950fd3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:54.972158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=100 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:55.025863 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1118/2249] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:09:55.036453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fbba71-28e3-4bf3-b7be-f67b8c12a2a1 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:55.037931 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-26fbba71-28e3-4bf3-b7be-f67b8c12a2a1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:55.044718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:55.106235 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fbba71-28e3-4bf3-b7be-f67b8c12a2a1 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:55.110025 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69d239-92e1-4e02-a5d1-cb40d168a13a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:55.110737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69d239-92e1-4e02-a5d1-cb40d168a13a service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:55.111022 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69d239-92e1-4e02-a5d1-cb40d168a13a service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:55.111297 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af69d239-92e1-4e02-a5d1-cb40d168a13a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:55.123368 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69d239-92e1-4e02-a5d1-cb40d168a13a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:55.124835 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af69d239-92e1-4e02-a5d1-cb40d168a13a service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:55.137090 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1132/2250] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:09:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:09:55.537895 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41e523-9fce-41a0-9d00-c5bf950fd3de None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:55.539891 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ba41e523-9fce-41a0-9d00-c5bf950fd3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:55.581365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba41e523-9fce-41a0-9d00-c5bf950fd3de None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:55.591095 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38320360-9222-4d6d-84f7-e9d1ddc831b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:55.593409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38320360-9222-4d6d-84f7-e9d1ddc831b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:55.593958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38320360-9222-4d6d-84f7-e9d1ddc831b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:55.594899 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38320360-9222-4d6d-84f7-e9d1ddc831b0 admin admin] PATH_INFO: `/v3/projects/a6dcd28d638144efa1a7e4955c8b4477` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:55.598238 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38320360-9222-4d6d-84f7-e9d1ddc831b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6dcd28d638144efa1a7e4955c8b4477)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:55.603304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38320360-9222-4d6d-84f7-e9d1ddc831b0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:55.669248 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-38320360-9222-4d6d-84f7-e9d1ddc831b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6dcd28d638144efa1a7e4955c8b4477', 'request_id': 'req-38320360-9222-4d6d-84f7-e9d1ddc831b0'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:55.687731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-38320360-9222-4d6d-84f7-e9d1ddc831b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6dcd28d638144efa1a7e4955c8b4477', 'request_id': 'req-38320360-9222-4d6d-84f7-e9d1ddc831b0'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:55.698744 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-38320360-9222-4d6d-84f7-e9d1ddc831b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6dcd28d638144efa1a7e4955c8b4477', 'request_id': 'req-38320360-9222-4d6d-84f7-e9d1ddc831b0'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:55.721819 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-38320360-9222-4d6d-84f7-e9d1ddc831b0 admin admin] The token cache is being invalidate because project a6dcd28d638144efa1a7e4955c8b4477 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:09:55.722335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-38320360-9222-4d6d-84f7-e9d1ddc831b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:55.723252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-38320360-9222-4d6d-84f7-e9d1ddc831b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:55.725300 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1119/2251] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:09:55 2022] DELETE /identity/v3/projects/a6dcd28d638144efa1a7e4955c8b4477 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:55.729498 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69d239-92e1-4e02-a5d1-cb40d168a13a service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:55.740664 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af69d239-92e1-4e02-a5d1-cb40d168a13a service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:55.819596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67d55e-de18-42af-bc3d-79f9c9834e71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:55.820612 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67d55e-de18-42af-bc3d-79f9c9834e71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:55.820897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67d55e-de18-42af-bc3d-79f9c9834e71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:55.821293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b67d55e-de18-42af-bc3d-79f9c9834e71 admin admin] PATH_INFO: `/v3/users/736705b33f034d828ecce1d13037f4ce` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:55.822544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67d55e-de18-42af-bc3d-79f9c9834e71 admin admin] RBAC: Authorizing `identity:delete_user(user_id=736705b33f034d828ecce1d13037f4ce)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:55.824693 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b67d55e-de18-42af-bc3d-79f9c9834e71 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:55.897838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-5b67d55e-de18-42af-bc3d-79f9c9834e71 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '736705b33f034d828ecce1d13037f4ce'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:55.918616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-5b67d55e-de18-42af-bc3d-79f9c9834e71 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '736705b33f034d828ecce1d13037f4ce'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:55.931263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-5b67d55e-de18-42af-bc3d-79f9c9834e71 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '736705b33f034d828ecce1d13037f4ce'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:55.942514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-5b67d55e-de18-42af-bc3d-79f9c9834e71 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '736705b33f034d828ecce1d13037f4ce'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:55.957283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-5b67d55e-de18-42af-bc3d-79f9c9834e71 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '736705b33f034d828ecce1d13037f4ce'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:55.965459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1133/2252] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:09:55 2022] DELETE /identity/v3/users/736705b33f034d828ecce1d13037f4ce => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:56.379219 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38320360-9222-4d6d-84f7-e9d1ddc831b0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:56.380669 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-38320360-9222-4d6d-84f7-e9d1ddc831b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:56.394222 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38320360-9222-4d6d-84f7-e9d1ddc831b0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:56.453169 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4773eda-f598-4250-9b7d-e6218335cb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:56.453965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4773eda-f598-4250-9b7d-e6218335cb7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:56.454254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4773eda-f598-4250-9b7d-e6218335cb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:56.454564 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4773eda-f598-4250-9b7d-e6218335cb7d admin admin] PATH_INFO: `/v3/projects/17d542a5136048cf8fc6310f300193f0` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:56.456252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4773eda-f598-4250-9b7d-e6218335cb7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=17d542a5136048cf8fc6310f300193f0)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:56.458153 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4773eda-f598-4250-9b7d-e6218335cb7d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:56.506961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:56.508586 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c4773eda-f598-4250-9b7d-e6218335cb7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '17d542a5136048cf8fc6310f300193f0', 'request_id': 'req-c4773eda-f598-4250-9b7d-e6218335cb7d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:56.516545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c4773eda-f598-4250-9b7d-e6218335cb7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '17d542a5136048cf8fc6310f300193f0', 'request_id': 'req-c4773eda-f598-4250-9b7d-e6218335cb7d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:56.525324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c4773eda-f598-4250-9b7d-e6218335cb7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '17d542a5136048cf8fc6310f300193f0', 'request_id': 'req-c4773eda-f598-4250-9b7d-e6218335cb7d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:56.539055 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c4773eda-f598-4250-9b7d-e6218335cb7d admin admin] The token cache is being invalidate because project 17d542a5136048cf8fc6310f300193f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:09:56.539330 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c4773eda-f598-4250-9b7d-e6218335cb7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:56.539991 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c4773eda-f598-4250-9b7d-e6218335cb7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:56.542020 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1120/2253] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:09:56 2022] DELETE /identity/v3/projects/17d542a5136048cf8fc6310f300193f0 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:56.546113 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67d55e-de18-42af-bc3d-79f9c9834e71 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:56.547590 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5b67d55e-de18-42af-bc3d-79f9c9834e71 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:56.560824 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b67d55e-de18-42af-bc3d-79f9c9834e71 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:56.589286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:56.626855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a48d23-1a62-4259-9ad5-6da82199f7bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:56.628307 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a48d23-1a62-4259-9ad5-6da82199f7bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:56.628590 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a48d23-1a62-4259-9ad5-6da82199f7bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:56.628863 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a48d23-1a62-4259-9ad5-6da82199f7bf admin admin] PATH_INFO: `/v3/users/e427cea9ba584432a8cde6034e6f686f` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:56.630056 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a48d23-1a62-4259-9ad5-6da82199f7bf admin admin] RBAC: Authorizing `identity:delete_user(user_id=e427cea9ba584432a8cde6034e6f686f)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:56.632027 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a48d23-1a62-4259-9ad5-6da82199f7bf admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:56.680755 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-34a48d23-1a62-4259-9ad5-6da82199f7bf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e427cea9ba584432a8cde6034e6f686f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:56.695191 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-34a48d23-1a62-4259-9ad5-6da82199f7bf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e427cea9ba584432a8cde6034e6f686f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:56.703021 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-34a48d23-1a62-4259-9ad5-6da82199f7bf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e427cea9ba584432a8cde6034e6f686f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:56.703720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-34a48d23-1a62-4259-9ad5-6da82199f7bf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e427cea9ba584432a8cde6034e6f686f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:56.716104 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-34a48d23-1a62-4259-9ad5-6da82199f7bf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e427cea9ba584432a8cde6034e6f686f'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:56.724176 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1134/2254] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:09:56 2022] DELETE /identity/v3/users/e427cea9ba584432a8cde6034e6f686f => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:57.013516 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4773eda-f598-4250-9b7d-e6218335cb7d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:57.015019 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c4773eda-f598-4250-9b7d-e6218335cb7d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:57.032012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4773eda-f598-4250-9b7d-e6218335cb7d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:57.101381 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765b693-129f-4009-9e95-1bc7fe0ad20e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:57.102156 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765b693-129f-4009-9e95-1bc7fe0ad20e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:57.102491 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765b693-129f-4009-9e95-1bc7fe0ad20e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:57.102775 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765b693-129f-4009-9e95-1bc7fe0ad20e admin admin] PATH_INFO: `/v3/projects/c1155ee336024a50a941b1649f624a98` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:57.104014 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e765b693-129f-4009-9e95-1bc7fe0ad20e admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1155ee336024a50a941b1649f624a98)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:57.105944 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e765b693-129f-4009-9e95-1bc7fe0ad20e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:57.154939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e765b693-129f-4009-9e95-1bc7fe0ad20e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1155ee336024a50a941b1649f624a98', 'request_id': 'req-e765b693-129f-4009-9e95-1bc7fe0ad20e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:57.160554 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e765b693-129f-4009-9e95-1bc7fe0ad20e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1155ee336024a50a941b1649f624a98', 'request_id': 'req-e765b693-129f-4009-9e95-1bc7fe0ad20e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:57.169347 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e765b693-129f-4009-9e95-1bc7fe0ad20e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1155ee336024a50a941b1649f624a98', 'request_id': 'req-e765b693-129f-4009-9e95-1bc7fe0ad20e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:57.181871 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e765b693-129f-4009-9e95-1bc7fe0ad20e admin admin] The token cache is being invalidate because project c1155ee336024a50a941b1649f624a98 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:09:57.182160 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e765b693-129f-4009-9e95-1bc7fe0ad20e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:57.182833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e765b693-129f-4009-9e95-1bc7fe0ad20e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:09:57.184401 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1121/2255] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:09:57 2022] DELETE /identity/v3/projects/c1155ee336024a50a941b1649f624a98 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:09:57.191282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96bf89-de57-4b98-b6c2-f76fca1b3574 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:57.191824 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96bf89-de57-4b98-b6c2-f76fca1b3574 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:57.191925 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad96bf89-de57-4b98-b6c2-f76fca1b3574 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:57.211049 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ad96bf89-de57-4b98-b6c2-f76fca1b3574 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:57.212378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ad96bf89-de57-4b98-b6c2-f76fca1b3574 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:57.277883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1135/2256] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:57.346522 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87a33b-3599-429b-8cfb-149ecb2c1d8d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:57.346827 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87a33b-3599-429b-8cfb-149ecb2c1d8d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:57.347114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df87a33b-3599-429b-8cfb-149ecb2c1d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:57.366246 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-df87a33b-3599-429b-8cfb-149ecb2c1d8d None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:57.367508 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-df87a33b-3599-429b-8cfb-149ecb2c1d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:57.375539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1122/2257] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:09:57.379821 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96bf89-de57-4b98-b6c2-f76fca1b3574 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:57.389589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad96bf89-de57-4b98-b6c2-f76fca1b3574 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:57.394261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0a4aa-10c4-49d3-b87e-b36359dc8aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:57.394959 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0a4aa-10c4-49d3-b87e-b36359dc8aa3 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:57.395247 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0a4aa-10c4-49d3-b87e-b36359dc8aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:57.395535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf0a4aa-10c4-49d3-b87e-b36359dc8aa3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:57.396273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0a4aa-10c4-49d3-b87e-b36359dc8aa3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:09:57.397226 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0a4aa-10c4-49d3-b87e-b36359dc8aa3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:57.398899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bf0a4aa-10c4-49d3-b87e-b36359dc8aa3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:57.411475 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1136/2258] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:09:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:09:57.415989 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87a33b-3599-429b-8cfb-149ecb2c1d8d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:57.426084 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df87a33b-3599-429b-8cfb-149ecb2c1d8d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:57.431215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aef52f-ed36-443e-b59f-14b96325edce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:57.431875 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aef52f-ed36-443e-b59f-14b96325edce admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:57.432162 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aef52f-ed36-443e-b59f-14b96325edce admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:57.432443 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75aef52f-ed36-443e-b59f-14b96325edce admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:57.433155 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aef52f-ed36-443e-b59f-14b96325edce admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:57.435134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75aef52f-ed36-443e-b59f-14b96325edce admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:57.461218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1123/2259] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:09:57 2022] POST /identity/v3/projects => generated 377 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:09:57.465436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0a4aa-10c4-49d3-b87e-b36359dc8aa3 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:57.477370 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bf0a4aa-10c4-49d3-b87e-b36359dc8aa3 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:57.511676 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:57.542230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60206333-5081-480e-ad2a-8f84df9032e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:57.542924 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60206333-5081-480e-ad2a-8f84df9032e5 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:57.543239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60206333-5081-480e-ad2a-8f84df9032e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:57.543520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60206333-5081-480e-ad2a-8f84df9032e5 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:57.544239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60206333-5081-480e-ad2a-8f84df9032e5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:57.546195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60206333-5081-480e-ad2a-8f84df9032e5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:57.571480 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1137/2260] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:09:57 2022] POST /identity/v3/users => generated 289 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:09:57.573615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aef52f-ed36-443e-b59f-14b96325edce admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:57.584472 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75aef52f-ed36-443e-b59f-14b96325edce admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:57.589218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deeff29-7d7b-4955-9ef6-715c98ef96e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:57.589896 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deeff29-7d7b-4955-9ef6-715c98ef96e1 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:57.590190 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deeff29-7d7b-4955-9ef6-715c98ef96e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:57.590509 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6deeff29-7d7b-4955-9ef6-715c98ef96e1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:57.591265 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deeff29-7d7b-4955-9ef6-715c98ef96e1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:57.592873 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6deeff29-7d7b-4955-9ef6-715c98ef96e1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:57.604526 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1124/2261] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:09:57.608721 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60206333-5081-480e-ad2a-8f84df9032e5 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:57.619781 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60206333-5081-480e-ad2a-8f84df9032e5 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:57.624559 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-601509d5-4237-4291-8ce0-850f95860a89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:57.625222 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601509d5-4237-4291-8ce0-850f95860a89 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:57.625510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601509d5-4237-4291-8ce0-850f95860a89 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:57.625792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-601509d5-4237-4291-8ce0-850f95860a89 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:57.626579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601509d5-4237-4291-8ce0-850f95860a89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:57.628335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-601509d5-4237-4291-8ce0-850f95860a89 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:57.638927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1138/2262] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:09:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:09:57.643793 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deeff29-7d7b-4955-9ef6-715c98ef96e1 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:57.654336 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6deeff29-7d7b-4955-9ef6-715c98ef96e1 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:57.660335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2907f70-3dbb-414c-b74c-47b80ebe01d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:57.661317 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2907f70-3dbb-414c-b74c-47b80ebe01d7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:57.661517 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2907f70-3dbb-414c-b74c-47b80ebe01d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:57.661797 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2907f70-3dbb-414c-b74c-47b80ebe01d7 admin admin] PATH_INFO: `/v3/projects/af635f3819c04b288903644eb8e18743/users/6d2d231465da45f683b91d7b8e7a9429/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:57.677436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2907f70-3dbb-414c-b74c-47b80ebe01d7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=af635f3819c04b288903644eb8e18743, user_id=6d2d231465da45f683b91d7b8e7a9429, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:57.680758 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2907f70-3dbb-414c-b74c-47b80ebe01d7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:57.694287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1125/2263] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:09:57 2022] PUT /identity/v3/projects/af635f3819c04b288903644eb8e18743/users/6d2d231465da45f683b91d7b8e7a9429/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:09:57.706204 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f30863-03a0-4a66-ab7c-3b653bb3887d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:57.706517 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f30863-03a0-4a66-ab7c-3b653bb3887d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:57.706801 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03f30863-03a0-4a66-ab7c-3b653bb3887d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:57.734483 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-03f30863-03a0-4a66-ab7c-3b653bb3887d None None] MFA Rules not processed for user `6d2d231465da45f683b91d7b8e7a9429`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:09:57.736024 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-03f30863-03a0-4a66-ab7c-3b653bb3887d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:57.802270 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1139/2264] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:09:57 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:09:57.814362 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2907f70-3dbb-414c-b74c-47b80ebe01d7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:57.816112 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b2907f70-3dbb-414c-b74c-47b80ebe01d7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:57.821485 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:09:57.896188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2907f70-3dbb-414c-b74c-47b80ebe01d7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:57.900145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8cd4e-8f0a-44f0-b001-980e914443bd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:57.900815 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8cd4e-8f0a-44f0-b001-980e914443bd service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:57.901127 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8cd4e-8f0a-44f0-b001-980e914443bd service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:57.901408 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39b8cd4e-8f0a-44f0-b001-980e914443bd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:57.912588 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8cd4e-8f0a-44f0-b001-980e914443bd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:57.914076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39b8cd4e-8f0a-44f0-b001-980e914443bd service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:57.923739 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1126/2265] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:09:57 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:09:58.601677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f30863-03a0-4a66-ab7c-3b653bb3887d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:09:58.603115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-03f30863-03a0-4a66-ab7c-3b653bb3887d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:58.682603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03f30863-03a0-4a66-ab7c-3b653bb3887d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:09:58.689586 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12510f23-fefb-428d-932a-6948522af09f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee29b7f1cea64d6781e1df5c8021363f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:09:58.690329 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12510f23-fefb-428d-932a-6948522af09f service placement] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:09:58.690751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12510f23-fefb-428d-932a-6948522af09f service placement] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:09:58.690923 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12510f23-fefb-428d-932a-6948522af09f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:09:58.692882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-12510f23-fefb-428d-932a-6948522af09f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:09:58.715248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12510f23-fefb-428d-932a-6948522af09f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:09:58.715248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12510f23-fefb-428d-932a-6948522af09f service placement] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:09:58.728169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1140/2266] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:09:58 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:10:07.919332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=25 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:10:08.728240 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=83,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:10:23.709429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8cd4e-8f0a-44f0-b001-980e914443bd service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:10:23.710895 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-39b8cd4e-8f0a-44f0-b001-980e914443bd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:10:23.768012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39b8cd4e-8f0a-44f0-b001-980e914443bd service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:10:23.771712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623c38ad-fa0d-43f7-ad5e-fec7ec0a8782 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:10:23.772371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c38ad-fa0d-43f7-ad5e-fec7ec0a8782 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:10:23.772658 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c38ad-fa0d-43f7-ad5e-fec7ec0a8782 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:10:23.772938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c38ad-fa0d-43f7-ad5e-fec7ec0a8782 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:10:23.774753 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-623c38ad-fa0d-43f7-ad5e-fec7ec0a8782 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:10:23.840527 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c38ad-fa0d-43f7-ad5e-fec7ec0a8782 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:10:23.842025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623c38ad-fa0d-43f7-ad5e-fec7ec0a8782 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:10:23.851794 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1127/2267] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:10:23 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:10:27.346447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12510f23-fefb-428d-932a-6948522af09f service placement] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:10:27.424501 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12510f23-fefb-428d-932a-6948522af09f service placement] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:10:27.433182 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c95e5-5f85-49e4-90f9-10e47a19f354 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:10:27.434835 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c95e5-5f85-49e4-90f9-10e47a19f354 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:10:27.435371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c95e5-5f85-49e4-90f9-10e47a19f354 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:10:27.435880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-866c95e5-5f85-49e4-90f9-10e47a19f354 admin admin] PATH_INFO: `/v3/users/6d2d231465da45f683b91d7b8e7a9429` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:10:27.438160 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c95e5-5f85-49e4-90f9-10e47a19f354 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6d2d231465da45f683b91d7b8e7a9429)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:10:27.442581 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-866c95e5-5f85-49e4-90f9-10e47a19f354 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:10:27.510902 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-866c95e5-5f85-49e4-90f9-10e47a19f354 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6d2d231465da45f683b91d7b8e7a9429'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:10:27.530493 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-866c95e5-5f85-49e4-90f9-10e47a19f354 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6d2d231465da45f683b91d7b8e7a9429'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:10:27.542897 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-866c95e5-5f85-49e4-90f9-10e47a19f354 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6d2d231465da45f683b91d7b8e7a9429'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:10:27.544269 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-866c95e5-5f85-49e4-90f9-10e47a19f354 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6d2d231465da45f683b91d7b8e7a9429'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:10:27.554724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-866c95e5-5f85-49e4-90f9-10e47a19f354 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6d2d231465da45f683b91d7b8e7a9429'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:10:27.564735 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1141/2268] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:10:27 2022] DELETE /identity/v3/users/6d2d231465da45f683b91d7b8e7a9429 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:10:27.566832 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf8a30-fd90-4502-a1f5-b12b442a51d1 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:10:27.567329 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf8a30-fd90-4502-a1f5-b12b442a51d1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:10:27.567413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96cf8a30-fd90-4502-a1f5-b12b442a51d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:10:27.586864 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-96cf8a30-fd90-4502-a1f5-b12b442a51d1 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:10:27.588115 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-96cf8a30-fd90-4502-a1f5-b12b442a51d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:10:27.650145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1128/2269] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:10:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:10:27.659794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c95e5-5f85-49e4-90f9-10e47a19f354 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:10:27.661243 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-866c95e5-5f85-49e4-90f9-10e47a19f354 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:10:27.723345 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-866c95e5-5f85-49e4-90f9-10e47a19f354 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:10:27.727186 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dd17488-8d6d-4aad-afc8-a93959d59ee6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:10:27.727847 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd17488-8d6d-4aad-afc8-a93959d59ee6 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:10:27.728132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd17488-8d6d-4aad-afc8-a93959d59ee6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:10:27.728407 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dd17488-8d6d-4aad-afc8-a93959d59ee6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:10:27.738643 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd17488-8d6d-4aad-afc8-a93959d59ee6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:10:27.740084 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dd17488-8d6d-4aad-afc8-a93959d59ee6 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:10:27.750823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1142/2270] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:10:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:10:28.178124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf8a30-fd90-4502-a1f5-b12b442a51d1 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:10:28.180909 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-96cf8a30-fd90-4502-a1f5-b12b442a51d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:10:28.207151 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96cf8a30-fd90-4502-a1f5-b12b442a51d1 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:10:28.213058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3173ce5a-23be-446c-90af-3208aa7b266d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:10:28.214332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173ce5a-23be-446c-90af-3208aa7b266d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:10:28.214943 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173ce5a-23be-446c-90af-3208aa7b266d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:10:28.215431 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3173ce5a-23be-446c-90af-3208aa7b266d admin admin] PATH_INFO: `/v3/projects/af635f3819c04b288903644eb8e18743` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:10:28.217170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173ce5a-23be-446c-90af-3208aa7b266d admin admin] RBAC: Authorizing `identity:delete_project(project_id=af635f3819c04b288903644eb8e18743)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:10:28.220170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3173ce5a-23be-446c-90af-3208aa7b266d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:10:28.293909 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:10:28.298517 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3173ce5a-23be-446c-90af-3208aa7b266d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'af635f3819c04b288903644eb8e18743', 'request_id': 'req-3173ce5a-23be-446c-90af-3208aa7b266d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:10:28.316006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3173ce5a-23be-446c-90af-3208aa7b266d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'af635f3819c04b288903644eb8e18743', 'request_id': 'req-3173ce5a-23be-446c-90af-3208aa7b266d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:10:28.326827 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3173ce5a-23be-446c-90af-3208aa7b266d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'af635f3819c04b288903644eb8e18743', 'request_id': 'req-3173ce5a-23be-446c-90af-3208aa7b266d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:10:28.346114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3173ce5a-23be-446c-90af-3208aa7b266d admin admin] The token cache is being invalidate because project af635f3819c04b288903644eb8e18743 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:10:28.346441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3173ce5a-23be-446c-90af-3208aa7b266d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:10:28.347111 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3173ce5a-23be-446c-90af-3208aa7b266d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:10:28.348666 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1129/2271] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:10:28 2022] DELETE /identity/v3/projects/af635f3819c04b288903644eb8e18743 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:10:32.256613 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfe60f-d402-4f57-a206-12f4076af381 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:10:32.257150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfe60f-d402-4f57-a206-12f4076af381 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:10:32.257243 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80bfe60f-d402-4f57-a206-12f4076af381 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:10:32.275565 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-80bfe60f-d402-4f57-a206-12f4076af381 None None] MFA Rules not processed for user `04e86369906d4633b02a5ca1bcc24b35`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:10:32.276798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-80bfe60f-d402-4f57-a206-12f4076af381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:10:32.294976 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:10:32.343040 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1143/2272] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:10:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:10:38.342338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=4,keystone:SELECT=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:10:42.335415 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=15 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:10:42.659085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af736c80-9d45-45e6-a92c-1506c81e2215 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:10:42.659085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af736c80-9d45-45e6-a92c-1506c81e2215 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:10:42.659754 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af736c80-9d45-45e6-a92c-1506c81e2215 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:10:42.678690 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-af736c80-9d45-45e6-a92c-1506c81e2215 None None] MFA Rules not processed for user `52076f59f68f4d77a0dadbf7a98d2b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:10:42.679915 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-af736c80-9d45-45e6-a92c-1506c81e2215 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:10:42.743175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1130/2273] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 11 14:10:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:10:42.786972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfe60f-d402-4f57-a206-12f4076af381 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:10:42.788371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-80bfe60f-d402-4f57-a206-12f4076af381 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:10:42.860424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80bfe60f-d402-4f57-a206-12f4076af381 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:10:42.865603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c48531-3e22-453a-a4c6-b6151222a0d4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c1369033a44c1c81d4c1c9ff2bbedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:10:42.866252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c48531-3e22-453a-a4c6-b6151222a0d4 service swift] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:10:42.866569 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c48531-3e22-453a-a4c6-b6151222a0d4 service swift] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:10:42.866870 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c48531-3e22-453a-a4c6-b6151222a0d4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:10:42.875110 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c48531-3e22-453a-a4c6-b6151222a0d4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:10:42.877248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c48531-3e22-453a-a4c6-b6151222a0d4 service swift] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:10:42.886174 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1144/2274] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 11 14:10:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 14:10:52.736743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=25 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:10:52.882570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=32 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:11:00.126896 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42ce13-df3b-4526-9769-b7277ed490b9 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:11:00.127497 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42ce13-df3b-4526-9769-b7277ed490b9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:11:00.127497 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f42ce13-df3b-4526-9769-b7277ed490b9 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:11:00.128731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1131/2275] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 11 14:11:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:11:00.134952 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac07c5-d5aa-4a5a-8bd6-8ac76dc844e2 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:11:00.135520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac07c5-d5aa-4a5a-8bd6-8ac76dc844e2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:11:00.135520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ac07c5-d5aa-4a5a-8bd6-8ac76dc844e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:11:00.155264 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-90ac07c5-d5aa-4a5a-8bd6-8ac76dc844e2 None None] MFA Rules not processed for user `ef5dac3f311e43809a46995bab56e780`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:11:00.156565 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-90ac07c5-d5aa-4a5a-8bd6-8ac76dc844e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:11:00.230791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1145/2276] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 11 14:11:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:11:00.242935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42ce13-df3b-4526-9769-b7277ed490b9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:11:00.244820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2f42ce13-df3b-4526-9769-b7277ed490b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:11:00.316296 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f42ce13-df3b-4526-9769-b7277ed490b9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:11:00.321184 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b8929b2-0309-4ea4-a686-7272f321302e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f80d59fec4994e689e857ba2087d3a23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:11:00.321863 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8929b2-0309-4ea4-a686-7272f321302e service glance] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:11:00.322134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8929b2-0309-4ea4-a686-7272f321302e service glance] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:11:00.322961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b8929b2-0309-4ea4-a686-7272f321302e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:11:00.334054 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8929b2-0309-4ea4-a686-7272f321302e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:11:00.337825 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b8929b2-0309-4ea4-a686-7272f321302e service glance] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:11:00.347433 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1132/2277] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:11:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:11:00.430331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a83f6-168f-4299-9590-a762a1331e58 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:11:00.430792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a83f6-168f-4299-9590-a762a1331e58 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:11:00.431171 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61a83f6-168f-4299-9590-a762a1331e58 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:11:00.432693 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1146/2278] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 11 14:11:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:11:00.439794 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131530c8-6914-4837-b90f-b98e018f75ef None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:11:00.440249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131530c8-6914-4837-b90f-b98e018f75ef None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:11:00.440661 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131530c8-6914-4837-b90f-b98e018f75ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:11:00.461394 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-131530c8-6914-4837-b90f-b98e018f75ef None None] MFA Rules not processed for user `ce1c85ee44ef45d2b631e51112b5ea0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:11:00.462778 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-131530c8-6914-4837-b90f-b98e018f75ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:11:00.515647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1133/2279] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 11 14:11:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:11:00.528199 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a83f6-168f-4299-9590-a762a1331e58 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:11:00.529891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b61a83f6-168f-4299-9590-a762a1331e58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:11:00.544353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61a83f6-168f-4299-9590-a762a1331e58 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:11:00.548003 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2940c2-15de-4371-aa49-288a1246f563 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:11:00.548696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2940c2-15de-4371-aa49-288a1246f563 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:11:00.548982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2940c2-15de-4371-aa49-288a1246f563 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:11:00.549262 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2940c2-15de-4371-aa49-288a1246f563 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:11:00.557032 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2940c2-15de-4371-aa49-288a1246f563 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:11:00.558509 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2940c2-15de-4371-aa49-288a1246f563 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:11:00.567035 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1147/2280] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:11:00 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:11:00.649932 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89caab5f-0357-4c92-a60a-9571a2623956 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:11:00.650223 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89caab5f-0357-4c92-a60a-9571a2623956 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:11:00.650553 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89caab5f-0357-4c92-a60a-9571a2623956 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:11:00.651937 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1134/2281] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 11 14:11:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:11:00.657362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e4e75-fbfb-41ef-a38e-f408956fd689 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:11:00.658291 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e4e75-fbfb-41ef-a38e-f408956fd689 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:11:00.658667 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-499e4e75-fbfb-41ef-a38e-f408956fd689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:11:00.679750 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-499e4e75-fbfb-41ef-a38e-f408956fd689 None None] MFA Rules not processed for user `ef5dac3f311e43809a46995bab56e780`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:11:00.680979 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-499e4e75-fbfb-41ef-a38e-f408956fd689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:11:00.695856 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1148/2282] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 11 14:11:00 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:11:00.706176 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89caab5f-0357-4c92-a60a-9571a2623956 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:11:00.717656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89caab5f-0357-4c92-a60a-9571a2623956 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:11:00.721888 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075a709-b970-4e32-9b8c-55438e5d4ec8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:11:00.722555 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075a709-b970-4e32-9b8c-55438e5d4ec8 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:11:00.722843 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075a709-b970-4e32-9b8c-55438e5d4ec8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:11:00.723123 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075a709-b970-4e32-9b8c-55438e5d4ec8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:11:00.734588 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075a709-b970-4e32-9b8c-55438e5d4ec8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:11:00.736598 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f075a709-b970-4e32-9b8c-55438e5d4ec8 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:11:00.747900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1135/2283] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:11:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:11:03.885065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957e340-9d7c-4100-a225-c2bf92b33ec4 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:11:03.885531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957e340-9d7c-4100-a225-c2bf92b33ec4 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:11:03.885665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d957e340-9d7c-4100-a225-c2bf92b33ec4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:11:03.903626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-d957e340-9d7c-4100-a225-c2bf92b33ec4 None None] MFA Rules not processed for user `52076f59f68f4d77a0dadbf7a98d2b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:11:03.904760 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d957e340-9d7c-4100-a225-c2bf92b33ec4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:11:03.912808 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1149/2284] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 11 14:11:03 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:11:03.925644 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075a709-b970-4e32-9b8c-55438e5d4ec8 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:11:03.937145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075a709-b970-4e32-9b8c-55438e5d4ec8 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:11:03.942134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e8d41-7d62-43b4-9b19-0ebf4dd21ce0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c1369033a44c1c81d4c1c9ff2bbedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:11:03.942848 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e8d41-7d62-43b4-9b19-0ebf4dd21ce0 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:11:03.943119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e8d41-7d62-43b4-9b19-0ebf4dd21ce0 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:11:03.943404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0e8d41-7d62-43b4-9b19-0ebf4dd21ce0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:11:03.951553 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e8d41-7d62-43b4-9b19-0ebf4dd21ce0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:11:03.953132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0e8d41-7d62-43b4-9b19-0ebf4dd21ce0 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:11:03.961841 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1136/2285] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 11 14:11:03 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 14:11:13.898228 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=43 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:11:13.958412 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=61 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:11:35.554647 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc47d7-bc77-45bd-8638-62032b9bf3ac None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:11:35.555213 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc47d7-bc77-45bd-8638-62032b9bf3ac None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:11:35.555213 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfc47d7-bc77-45bd-8638-62032b9bf3ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:11:35.575813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-bcfc47d7-bc77-45bd-8638-62032b9bf3ac None None] MFA Rules not processed for user `52076f59f68f4d77a0dadbf7a98d2b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:11:35.576868 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bcfc47d7-bc77-45bd-8638-62032b9bf3ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:11:35.584981 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1150/2286] 10.222.0.49 () {58 vars in 967 bytes} [Mon Jul 11 14:11:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:11:35.599419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e8d41-7d62-43b4-9b19-0ebf4dd21ce0 service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:11:35.611042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0e8d41-7d62-43b4-9b19-0ebf4dd21ce0 service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:11:35.616390 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02760f7c-42f3-40b7-ad22-11f4568f926b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c1369033a44c1c81d4c1c9ff2bbedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:11:35.617076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02760f7c-42f3-40b7-ad22-11f4568f926b service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:11:35.617364 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02760f7c-42f3-40b7-ad22-11f4568f926b service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:11:35.617650 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02760f7c-42f3-40b7-ad22-11f4568f926b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:11:35.627554 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02760f7c-42f3-40b7-ad22-11f4568f926b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:11:35.629215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02760f7c-42f3-40b7-ad22-11f4568f926b service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:11:35.639573 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1137/2287] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 11 14:11:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 14:11:45.570150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:11:45.636387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=6 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:11:51.238703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc47d7-bc77-45bd-8638-62032b9bf3ac None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:11:51.248228 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfc47d7-bc77-45bd-8638-62032b9bf3ac None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:11:51.257185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146d316b-6d80-4736-b381-47a0c09a5b75 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:11:51.257859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d316b-6d80-4736-b381-47a0c09a5b75 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:11:51.258147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d316b-6d80-4736-b381-47a0c09a5b75 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:11:51.258465 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146d316b-6d80-4736-b381-47a0c09a5b75 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:11:51.260327 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-146d316b-6d80-4736-b381-47a0c09a5b75 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:11:51.274286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d316b-6d80-4736-b381-47a0c09a5b75 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:11:51.275788 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146d316b-6d80-4736-b381-47a0c09a5b75 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:11:51.292181 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1151/2288] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:11:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 54 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:11:52.466576 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02760f7c-42f3-40b7-ad22-11f4568f926b service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:11:52.477105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02760f7c-42f3-40b7-ad22-11f4568f926b service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:11:52.480746 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586aa561-978a-4652-8096-16f933aaa21e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:11:52.481415 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586aa561-978a-4652-8096-16f933aaa21e service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:11:52.481707 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586aa561-978a-4652-8096-16f933aaa21e service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:11:52.481991 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586aa561-978a-4652-8096-16f933aaa21e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:11:52.483825 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-586aa561-978a-4652-8096-16f933aaa21e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:11:52.497627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586aa561-978a-4652-8096-16f933aaa21e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:11:52.499065 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586aa561-978a-4652-8096-16f933aaa21e service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:11:52.510014 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1138/2289] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:11:52 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:11:59.670752 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1aed93-178c-46b0-90d0-bd1ef0c6fbe4 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:11:59.671835 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1aed93-178c-46b0-90d0-bd1ef0c6fbe4 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:11:59.671835 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb1aed93-178c-46b0-90d0-bd1ef0c6fbe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:11:59.691077 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-eb1aed93-178c-46b0-90d0-bd1ef0c6fbe4 None None] MFA Rules not processed for user `cf6cc756845148ee8fce9021425cb7cb`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:11:59.692329 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-eb1aed93-178c-46b0-90d0-bd1ef0c6fbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:11:59.753816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1152/2290] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:11:59 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:12:02.476326 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586aa561-978a-4652-8096-16f933aaa21e service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:12:02.477880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-586aa561-978a-4652-8096-16f933aaa21e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:12:02.491327 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586aa561-978a-4652-8096-16f933aaa21e service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:12:02.496905 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cffd79-172e-498e-aba5-92ed25384d2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:12:02.497549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cffd79-172e-498e-aba5-92ed25384d2b service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:12:02.497839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cffd79-172e-498e-aba5-92ed25384d2b service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:12:02.498122 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14cffd79-172e-498e-aba5-92ed25384d2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:12:02.499911 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-14cffd79-172e-498e-aba5-92ed25384d2b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:12:02.510937 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cffd79-172e-498e-aba5-92ed25384d2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:12:02.512531 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14cffd79-172e-498e-aba5-92ed25384d2b service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:12:02.523050 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1139/2291] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:12:02 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:12:09.744065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=31 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:12:12.518998 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=12 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:12:22.262181 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1aed93-178c-46b0-90d0-bd1ef0c6fbe4 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:12:22.263734 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-eb1aed93-178c-46b0-90d0-bd1ef0c6fbe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:12:22.279822 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb1aed93-178c-46b0-90d0-bd1ef0c6fbe4 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:12:22.285899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81f97f-b898-445e-b18c-cbbe785c78fc service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:12:22.286567 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81f97f-b898-445e-b18c-cbbe785c78fc service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:12:22.286852 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81f97f-b898-445e-b18c-cbbe785c78fc service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:12:22.287133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac81f97f-b898-445e-b18c-cbbe785c78fc service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:12:22.288858 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ac81f97f-b898-445e-b18c-cbbe785c78fc service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:12:22.349085 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81f97f-b898-445e-b18c-cbbe785c78fc service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:12:22.350727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac81f97f-b898-445e-b18c-cbbe785c78fc service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:12:22.360277 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1153/2292] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:12:22 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:12:30.684796 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cffd79-172e-498e-aba5-92ed25384d2b service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:12:30.696225 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14cffd79-172e-498e-aba5-92ed25384d2b service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:12:30.702063 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330189ab-910e-45f6-a908-8d4a5d7b6f63 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:12:30.702975 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330189ab-910e-45f6-a908-8d4a5d7b6f63 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:12:30.703117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330189ab-910e-45f6-a908-8d4a5d7b6f63 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:12:30.703285 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330189ab-910e-45f6-a908-8d4a5d7b6f63 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:12:30.705071 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-330189ab-910e-45f6-a908-8d4a5d7b6f63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:12:30.715833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330189ab-910e-45f6-a908-8d4a5d7b6f63 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:12:30.717383 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330189ab-910e-45f6-a908-8d4a5d7b6f63 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:12:30.726810 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1140/2293] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:12:30 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:12:32.356210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=28 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:12:33.968336 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81f97f-b898-445e-b18c-cbbe785c78fc service nova] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:12:33.982072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac81f97f-b898-445e-b18c-cbbe785c78fc service nova] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:12:33.997989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5169cdcc-014e-4fc6-95f2-d7c715c739d8 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:12:34.000775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169cdcc-014e-4fc6-95f2-d7c715c739d8 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:12:34.001207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169cdcc-014e-4fc6-95f2-d7c715c739d8 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:12:34.001556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5169cdcc-014e-4fc6-95f2-d7c715c739d8 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:12:34.004713 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5169cdcc-014e-4fc6-95f2-d7c715c739d8 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:12:34.022210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5169cdcc-014e-4fc6-95f2-d7c715c739d8 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:12:34.024513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5169cdcc-014e-4fc6-95f2-d7c715c739d8 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:12:34.037841 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1154/2294] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:12:33 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 71 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:12:40.722630 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=6 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:12:42.091560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330189ab-910e-45f6-a908-8d4a5d7b6f63 service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:12:42.094065 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-330189ab-910e-45f6-a908-8d4a5d7b6f63 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:12:42.115877 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330189ab-910e-45f6-a908-8d4a5d7b6f63 service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:12:42.121314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625d9f4-7935-4ec9-9188-f2bcbac2c1c8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:12:42.122569 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625d9f4-7935-4ec9-9188-f2bcbac2c1c8 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:12:42.123122 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625d9f4-7935-4ec9-9188-f2bcbac2c1c8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:12:42.123648 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c625d9f4-7935-4ec9-9188-f2bcbac2c1c8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:12:42.127683 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c625d9f4-7935-4ec9-9188-f2bcbac2c1c8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:12:42.142223 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625d9f4-7935-4ec9-9188-f2bcbac2c1c8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:12:42.143701 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c625d9f4-7935-4ec9-9188-f2bcbac2c1c8 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:12:42.156254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1141/2295] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:12:42 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:12:44.032092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=6 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:12:44.837059 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cdd34b-e9bb-4301-8a2a-1b316c7f3325 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:12:44.837408 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cdd34b-e9bb-4301-8a2a-1b316c7f3325 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:12:44.837699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90cdd34b-e9bb-4301-8a2a-1b316c7f3325 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:12:44.858079 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-90cdd34b-e9bb-4301-8a2a-1b316c7f3325 None None] MFA Rules not processed for user `04e86369906d4633b02a5ca1bcc24b35`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:12:44.860433 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-90cdd34b-e9bb-4301-8a2a-1b316c7f3325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:12:44.867962 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1155/2296] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:12:44 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:12:45.199416 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625d9f4-7935-4ec9-9188-f2bcbac2c1c8 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:12:45.208526 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c625d9f4-7935-4ec9-9188-f2bcbac2c1c8 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:12:45.212273 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f6750-056c-463e-828c-933c19d7f2e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:12:45.212990 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f6750-056c-463e-828c-933c19d7f2e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:12:45.213301 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f6750-056c-463e-828c-933c19d7f2e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:12:45.213623 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0f6750-056c-463e-828c-933c19d7f2e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:12:45.215646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8e0f6750-056c-463e-828c-933c19d7f2e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:12:45.226139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f6750-056c-463e-828c-933c19d7f2e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:12:45.227624 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0f6750-056c-463e-828c-933c19d7f2e7 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:12:45.237580 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1142/2297] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:12:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:12:45.579552 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cdd34b-e9bb-4301-8a2a-1b316c7f3325 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:12:45.581735 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-90cdd34b-e9bb-4301-8a2a-1b316c7f3325 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:12:45.692406 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90cdd34b-e9bb-4301-8a2a-1b316c7f3325 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:12:45.698329 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e96cc-4435-4ba4-b903-d364d0749c98 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee29b7f1cea64d6781e1df5c8021363f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:12:45.699012 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e96cc-4435-4ba4-b903-d364d0749c98 service placement] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:12:45.699294 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e96cc-4435-4ba4-b903-d364d0749c98 service placement] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:12:45.699633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e96cc-4435-4ba4-b903-d364d0749c98 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:12:45.701607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-df9e96cc-4435-4ba4-b903-d364d0749c98 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:12:45.715796 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e96cc-4435-4ba4-b903-d364d0749c98 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:12:45.717429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e96cc-4435-4ba4-b903-d364d0749c98 service placement] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:12:45.729522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1156/2298] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:12:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:12:53.973317 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f6750-056c-463e-828c-933c19d7f2e7 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:12:53.996723 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0f6750-056c-463e-828c-933c19d7f2e7 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:12:54.003976 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2b5c0-c9d6-4ae7-acc7-86f44de26241 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:12:54.005069 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2b5c0-c9d6-4ae7-acc7-86f44de26241 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:12:54.005581 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2b5c0-c9d6-4ae7-acc7-86f44de26241 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:12:54.006242 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff2b5c0-c9d6-4ae7-acc7-86f44de26241 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:12:54.009127 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5ff2b5c0-c9d6-4ae7-acc7-86f44de26241 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:12:54.097613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2b5c0-c9d6-4ae7-acc7-86f44de26241 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:12:54.099450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff2b5c0-c9d6-4ae7-acc7-86f44de26241 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:12:54.111051 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1143/2299] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:12:53 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:12:55.395533 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e96cc-4435-4ba4-b903-d364d0749c98 service placement] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:12:55.411652 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e96cc-4435-4ba4-b903-d364d0749c98 service placement] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:12:55.419187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb94830-7e93-4afe-9916-0b9aa358eeb5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:12:55.420938 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb94830-7e93-4afe-9916-0b9aa358eeb5 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:12:55.421611 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb94830-7e93-4afe-9916-0b9aa358eeb5 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:12:55.422209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb94830-7e93-4afe-9916-0b9aa358eeb5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:12:55.426595 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ceb94830-7e93-4afe-9916-0b9aa358eeb5 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:12:55.461134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb94830-7e93-4afe-9916-0b9aa358eeb5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:12:55.464601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb94830-7e93-4afe-9916-0b9aa358eeb5 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:12:55.482667 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1157/2300] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:12:55 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:13:00.329566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2b5c0-c9d6-4ae7-acc7-86f44de26241 service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:00.343557 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff2b5c0-c9d6-4ae7-acc7-86f44de26241 service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:00.349788 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef56c49-bf75-4303-99c9-31ac23e559ef service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:00.350672 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef56c49-bf75-4303-99c9-31ac23e559ef service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:00.350969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef56c49-bf75-4303-99c9-31ac23e559ef service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:00.351246 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ef56c49-bf75-4303-99c9-31ac23e559ef service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:00.353388 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-6ef56c49-bf75-4303-99c9-31ac23e559ef service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:00.368340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef56c49-bf75-4303-99c9-31ac23e559ef service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:00.370535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ef56c49-bf75-4303-99c9-31ac23e559ef service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:00.384434 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1144/2301] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:13:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:13:05.476738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=41 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:13:07.847399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf97945-629d-4b08-9e1d-a93981ab4b3f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:07.848027 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf97945-629d-4b08-9e1d-a93981ab4b3f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:07.848027 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf97945-629d-4b08-9e1d-a93981ab4b3f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:07.866433 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9bf97945-629d-4b08-9e1d-a93981ab4b3f None None] MFA Rules not processed for user `98143abd2e3c47e3921184a1c5e05a89`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:13:07.867650 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9bf97945-629d-4b08-9e1d-a93981ab4b3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:07.874755 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1158/2302] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:13:07 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:13:08.239125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef56c49-bf75-4303-99c9-31ac23e559ef service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:08.248492 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ef56c49-bf75-4303-99c9-31ac23e559ef service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:08.252179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16777222-fca6-40b0-b2db-0b0829a44bb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:08.252976 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16777222-fca6-40b0-b2db-0b0829a44bb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:08.253268 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16777222-fca6-40b0-b2db-0b0829a44bb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:08.253555 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16777222-fca6-40b0-b2db-0b0829a44bb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:08.255570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-16777222-fca6-40b0-b2db-0b0829a44bb1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:08.266094 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16777222-fca6-40b0-b2db-0b0829a44bb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:08.267652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16777222-fca6-40b0-b2db-0b0829a44bb1 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:08.276117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1145/2303] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:13:08 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:13:17.861030 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:13:18.272980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=52 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:13:34.495928 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15440e-91a1-4070-9405-5ec1afba5f76 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:34.496354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15440e-91a1-4070-9405-5ec1afba5f76 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:34.496487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c15440e-91a1-4070-9405-5ec1afba5f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:34.514130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9c15440e-91a1-4070-9405-5ec1afba5f76 None None] MFA Rules not processed for user `52076f59f68f4d77a0dadbf7a98d2b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:13:34.515499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9c15440e-91a1-4070-9405-5ec1afba5f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:34.522837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1159/2304] 10.222.0.49 () {58 vars in 967 bytes} [Mon Jul 11 14:13:34 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:13:34.569033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16777222-fca6-40b0-b2db-0b0829a44bb1 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:34.581285 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16777222-fca6-40b0-b2db-0b0829a44bb1 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:34.586354 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b9a69-9414-4661-95eb-63ca3ba08182 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c1369033a44c1c81d4c1c9ff2bbedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:34.587004 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b9a69-9414-4661-95eb-63ca3ba08182 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:34.587283 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b9a69-9414-4661-95eb-63ca3ba08182 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:34.587559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-568b9a69-9414-4661-95eb-63ca3ba08182 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:34.596255 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b9a69-9414-4661-95eb-63ca3ba08182 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:34.597816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-568b9a69-9414-4661-95eb-63ca3ba08182 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:34.605846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1146/2305] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 11 14:13:34 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 14:13:38.175239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15440e-91a1-4070-9405-5ec1afba5f76 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:38.196364 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c15440e-91a1-4070-9405-5ec1afba5f76 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:38.202689 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3656dcff-bfe8-4dcf-b298-e34d9ce93363 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee29b7f1cea64d6781e1df5c8021363f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:38.203671 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656dcff-bfe8-4dcf-b298-e34d9ce93363 service placement] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:38.203965 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656dcff-bfe8-4dcf-b298-e34d9ce93363 service placement] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:38.204314 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3656dcff-bfe8-4dcf-b298-e34d9ce93363 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:38.206342 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-3656dcff-bfe8-4dcf-b298-e34d9ce93363 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:38.222528 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3656dcff-bfe8-4dcf-b298-e34d9ce93363 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:38.225110 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3656dcff-bfe8-4dcf-b298-e34d9ce93363 service placement] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:38.247340 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1160/2306] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:13:38 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 73 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:13:39.517491 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b9a69-9414-4661-95eb-63ca3ba08182 service swift] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:39.527774 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-568b9a69-9414-4661-95eb-63ca3ba08182 service swift] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:39.532104 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33372caf-b51d-4b75-934b-e9622111b7b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:39.532764 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33372caf-b51d-4b75-934b-e9622111b7b6 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:39.533052 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33372caf-b51d-4b75-934b-e9622111b7b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:39.533334 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33372caf-b51d-4b75-934b-e9622111b7b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:39.535209 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-33372caf-b51d-4b75-934b-e9622111b7b6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:39.604307 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33372caf-b51d-4b75-934b-e9622111b7b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:39.605784 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33372caf-b51d-4b75-934b-e9622111b7b6 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:39.616153 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1147/2307] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:13:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:13:41.738990 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053675c2-5215-495d-864e-1b659444aef2 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:41.739568 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053675c2-5215-495d-864e-1b659444aef2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:41.739568 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-053675c2-5215-495d-864e-1b659444aef2 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:41.741034 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1161/2308] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 11 14:13:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:13:41.755482 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f5779-2310-4b33-8eb4-ee0271118246 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:41.756033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f5779-2310-4b33-8eb4-ee0271118246 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:41.756294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e71f5779-2310-4b33-8eb4-ee0271118246 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:41.788096 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e71f5779-2310-4b33-8eb4-ee0271118246 None None] MFA Rules not processed for user `ee29b7f1cea64d6781e1df5c8021363f`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:13:41.789751 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e71f5779-2310-4b33-8eb4-ee0271118246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:41.803300 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1148/2309] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 11 14:13:41 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:13:41.824495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053675c2-5215-495d-864e-1b659444aef2 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:41.826497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-053675c2-5215-495d-864e-1b659444aef2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:41.850095 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-053675c2-5215-495d-864e-1b659444aef2 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:41.855923 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3906f63-8293-4629-92f6-52e55c18d864 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee29b7f1cea64d6781e1df5c8021363f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:41.857445 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3906f63-8293-4629-92f6-52e55c18d864 service placement] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:41.858009 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3906f63-8293-4629-92f6-52e55c18d864 service placement] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:41.858548 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3906f63-8293-4629-92f6-52e55c18d864 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:41.864704 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f5779-2310-4b33-8eb4-ee0271118246 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:41.866397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e71f5779-2310-4b33-8eb4-ee0271118246 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:41.885422 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3906f63-8293-4629-92f6-52e55c18d864 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:41.887990 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3906f63-8293-4629-92f6-52e55c18d864 service placement] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:41.889223 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e71f5779-2310-4b33-8eb4-ee0271118246 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:41.899982 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7184935-a771-4296-bfea-ce772d9adaa1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:41.900914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7184935-a771-4296-bfea-ce772d9adaa1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:41.901208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7184935-a771-4296-bfea-ce772d9adaa1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:41.901490 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7184935-a771-4296-bfea-ce772d9adaa1 admin admin] PATH_INFO: `/v3/users/365acac78e864078af1a3db4842bc149` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:41.901661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1162/2310] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:13:41 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 78 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:13:41.902944 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7184935-a771-4296-bfea-ce772d9adaa1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=365acac78e864078af1a3db4842bc149)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:41.905703 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7184935-a771-4296-bfea-ce772d9adaa1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:41.973206 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e7184935-a771-4296-bfea-ce772d9adaa1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '365acac78e864078af1a3db4842bc149'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:41.973916 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e7184935-a771-4296-bfea-ce772d9adaa1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '365acac78e864078af1a3db4842bc149'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:41.983100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e7184935-a771-4296-bfea-ce772d9adaa1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '365acac78e864078af1a3db4842bc149'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:41.983291 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e7184935-a771-4296-bfea-ce772d9adaa1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '365acac78e864078af1a3db4842bc149'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:42.007362 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e7184935-a771-4296-bfea-ce772d9adaa1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '365acac78e864078af1a3db4842bc149'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:42.029776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1149/2311] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:13:41 2022] DELETE /identity/v3/users/365acac78e864078af1a3db4842bc149 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:13:42.224289 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3906f63-8293-4629-92f6-52e55c18d864 service placement] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:42.238003 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3906f63-8293-4629-92f6-52e55c18d864 service placement] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:42.303723 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a880699-3bea-41c8-8b48-221979568c1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:42.304676 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a880699-3bea-41c8-8b48-221979568c1e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:42.304961 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a880699-3bea-41c8-8b48-221979568c1e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:42.305230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a880699-3bea-41c8-8b48-221979568c1e admin admin] PATH_INFO: `/v3/projects/32353b01d57e434eafdb5f253517ef93` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:42.306628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a880699-3bea-41c8-8b48-221979568c1e admin admin] RBAC: Authorizing `identity:delete_project(project_id=32353b01d57e434eafdb5f253517ef93)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:42.308589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a880699-3bea-41c8-8b48-221979568c1e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:42.363595 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4a880699-3bea-41c8-8b48-221979568c1e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '32353b01d57e434eafdb5f253517ef93', 'request_id': 'req-4a880699-3bea-41c8-8b48-221979568c1e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:42.372760 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4a880699-3bea-41c8-8b48-221979568c1e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '32353b01d57e434eafdb5f253517ef93', 'request_id': 'req-4a880699-3bea-41c8-8b48-221979568c1e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:42.378768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4a880699-3bea-41c8-8b48-221979568c1e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '32353b01d57e434eafdb5f253517ef93', 'request_id': 'req-4a880699-3bea-41c8-8b48-221979568c1e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:42.390506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4a880699-3bea-41c8-8b48-221979568c1e admin admin] The token cache is being invalidate because project 32353b01d57e434eafdb5f253517ef93 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:13:42.390794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4a880699-3bea-41c8-8b48-221979568c1e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:42.391555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4a880699-3bea-41c8-8b48-221979568c1e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:42.393605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1163/2312] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:13:42 2022] DELETE /identity/v3/projects/32353b01d57e434eafdb5f253517ef93 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:13:42.397174 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7184935-a771-4296-bfea-ce772d9adaa1 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:42.398584 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e7184935-a771-4296-bfea-ce772d9adaa1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:42.412672 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7184935-a771-4296-bfea-ce772d9adaa1 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:42.469488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:13:42.484484 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec1cf6-9bb2-41a7-98a6-1ecb05003b2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:42.485692 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec1cf6-9bb2-41a7-98a6-1ecb05003b2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:42.486153 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec1cf6-9bb2-41a7-98a6-1ecb05003b2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:42.486562 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ec1cf6-9bb2-41a7-98a6-1ecb05003b2d admin admin] PATH_INFO: `/v3/users/9aed963aa25e4d8faa6f715d74f9dd3f` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:42.488259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec1cf6-9bb2-41a7-98a6-1ecb05003b2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9aed963aa25e4d8faa6f715d74f9dd3f)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:42.490649 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ec1cf6-9bb2-41a7-98a6-1ecb05003b2d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:42.576695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f0ec1cf6-9bb2-41a7-98a6-1ecb05003b2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9aed963aa25e4d8faa6f715d74f9dd3f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:42.577389 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f0ec1cf6-9bb2-41a7-98a6-1ecb05003b2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9aed963aa25e4d8faa6f715d74f9dd3f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:42.588353 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f0ec1cf6-9bb2-41a7-98a6-1ecb05003b2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9aed963aa25e4d8faa6f715d74f9dd3f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:42.589310 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f0ec1cf6-9bb2-41a7-98a6-1ecb05003b2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9aed963aa25e4d8faa6f715d74f9dd3f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:42.605508 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-f0ec1cf6-9bb2-41a7-98a6-1ecb05003b2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9aed963aa25e4d8faa6f715d74f9dd3f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:42.620874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1150/2313] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:13:42 2022] DELETE /identity/v3/users/9aed963aa25e4d8faa6f715d74f9dd3f => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:13:42.967727 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a880699-3bea-41c8-8b48-221979568c1e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:42.969223 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4a880699-3bea-41c8-8b48-221979568c1e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:42.982290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a880699-3bea-41c8-8b48-221979568c1e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:43.043010 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:13:43.063390 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00c37d-5ce7-4910-a47a-b9cd6dc2b1ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:43.064292 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00c37d-5ce7-4910-a47a-b9cd6dc2b1ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:43.064581 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00c37d-5ce7-4910-a47a-b9cd6dc2b1ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:43.065055 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a00c37d-5ce7-4910-a47a-b9cd6dc2b1ec admin admin] PATH_INFO: `/v3/projects/b7d4d56298dc40c9b1fde368ed3f19f4` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:43.066382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00c37d-5ce7-4910-a47a-b9cd6dc2b1ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=b7d4d56298dc40c9b1fde368ed3f19f4)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:43.068351 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a00c37d-5ce7-4910-a47a-b9cd6dc2b1ec admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:43.137041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4a00c37d-5ce7-4910-a47a-b9cd6dc2b1ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b7d4d56298dc40c9b1fde368ed3f19f4', 'request_id': 'req-4a00c37d-5ce7-4910-a47a-b9cd6dc2b1ec'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:43.146560 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4a00c37d-5ce7-4910-a47a-b9cd6dc2b1ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b7d4d56298dc40c9b1fde368ed3f19f4', 'request_id': 'req-4a00c37d-5ce7-4910-a47a-b9cd6dc2b1ec'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:43.154523 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4a00c37d-5ce7-4910-a47a-b9cd6dc2b1ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b7d4d56298dc40c9b1fde368ed3f19f4', 'request_id': 'req-4a00c37d-5ce7-4910-a47a-b9cd6dc2b1ec'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:43.166805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4a00c37d-5ce7-4910-a47a-b9cd6dc2b1ec admin admin] The token cache is being invalidate because project b7d4d56298dc40c9b1fde368ed3f19f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:13:43.167092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4a00c37d-5ce7-4910-a47a-b9cd6dc2b1ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:43.167769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4a00c37d-5ce7-4910-a47a-b9cd6dc2b1ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:43.174830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1164/2314] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:13:42 2022] DELETE /identity/v3/projects/b7d4d56298dc40c9b1fde368ed3f19f4 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:13:43.211991 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec1cf6-9bb2-41a7-98a6-1ecb05003b2d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:43.215099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f0ec1cf6-9bb2-41a7-98a6-1ecb05003b2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:43.245052 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ec1cf6-9bb2-41a7-98a6-1ecb05003b2d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:43.310734 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00753cd9-2af2-495e-aa2b-949633406620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:43.311653 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00753cd9-2af2-495e-aa2b-949633406620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:43.311945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00753cd9-2af2-495e-aa2b-949633406620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:43.312224 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00753cd9-2af2-495e-aa2b-949633406620 admin admin] PATH_INFO: `/v3/users/cf6cc756845148ee8fce9021425cb7cb` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:43.313479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00753cd9-2af2-495e-aa2b-949633406620 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf6cc756845148ee8fce9021425cb7cb)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:43.315459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00753cd9-2af2-495e-aa2b-949633406620 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:43.374983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-00753cd9-2af2-495e-aa2b-949633406620 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf6cc756845148ee8fce9021425cb7cb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:43.375686 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-00753cd9-2af2-495e-aa2b-949633406620 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf6cc756845148ee8fce9021425cb7cb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:43.384266 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-00753cd9-2af2-495e-aa2b-949633406620 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf6cc756845148ee8fce9021425cb7cb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:43.385045 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-00753cd9-2af2-495e-aa2b-949633406620 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf6cc756845148ee8fce9021425cb7cb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:43.397833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=79,keystone:DELETE=19,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:13:43.400357 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-00753cd9-2af2-495e-aa2b-949633406620 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf6cc756845148ee8fce9021425cb7cb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:43.415085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1151/2315] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:13:43 2022] DELETE /identity/v3/users/cf6cc756845148ee8fce9021425cb7cb => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:13:43.595887 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00c37d-5ce7-4910-a47a-b9cd6dc2b1ec admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:43.597335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4a00c37d-5ce7-4910-a47a-b9cd6dc2b1ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:43.610961 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a00c37d-5ce7-4910-a47a-b9cd6dc2b1ec admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:43.667786 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3a4593e-c4b3-4337-b91c-00078d866e3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:43.668644 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4593e-c4b3-4337-b91c-00078d866e3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:43.668974 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4593e-c4b3-4337-b91c-00078d866e3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:43.669735 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a4593e-c4b3-4337-b91c-00078d866e3e admin admin] PATH_INFO: `/v3/projects/4b181bc391914f6abcea09b894b88473` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:43.670960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4593e-c4b3-4337-b91c-00078d866e3e admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b181bc391914f6abcea09b894b88473)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:43.672868 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3a4593e-c4b3-4337-b91c-00078d866e3e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:43.722652 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b3a4593e-c4b3-4337-b91c-00078d866e3e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b181bc391914f6abcea09b894b88473', 'request_id': 'req-b3a4593e-c4b3-4337-b91c-00078d866e3e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:43.730099 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b3a4593e-c4b3-4337-b91c-00078d866e3e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b181bc391914f6abcea09b894b88473', 'request_id': 'req-b3a4593e-c4b3-4337-b91c-00078d866e3e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:43.736153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b3a4593e-c4b3-4337-b91c-00078d866e3e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b181bc391914f6abcea09b894b88473', 'request_id': 'req-b3a4593e-c4b3-4337-b91c-00078d866e3e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:43.747500 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b3a4593e-c4b3-4337-b91c-00078d866e3e admin admin] The token cache is being invalidate because project 4b181bc391914f6abcea09b894b88473 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:13:43.747794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b3a4593e-c4b3-4337-b91c-00078d866e3e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:43.748485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b3a4593e-c4b3-4337-b91c-00078d866e3e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:43.752098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1165/2316] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:13:43 2022] DELETE /identity/v3/projects/4b181bc391914f6abcea09b894b88473 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:13:43.755202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06361c1-22d8-4507-9ab2-0f60a4a88a53 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:43.755502 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06361c1-22d8-4507-9ab2-0f60a4a88a53 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:43.755788 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f06361c1-22d8-4507-9ab2-0f60a4a88a53 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:43.774648 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f06361c1-22d8-4507-9ab2-0f60a4a88a53 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:13:43.775882 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f06361c1-22d8-4507-9ab2-0f60a4a88a53 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:43.838202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1152/2317] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:13:43.846711 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244a934-7510-462f-8775-65836395be73 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:43.846944 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244a934-7510-462f-8775-65836395be73 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:43.847220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9244a934-7510-462f-8775-65836395be73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:43.866811 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9244a934-7510-462f-8775-65836395be73 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:13:43.868008 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9244a934-7510-462f-8775-65836395be73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:43.876033 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1166/2318] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:13:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:13:43.880798 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06361c1-22d8-4507-9ab2-0f60a4a88a53 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:43.891324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f06361c1-22d8-4507-9ab2-0f60a4a88a53 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:43.896584 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243441ee-8659-45cb-9fae-3ec1b016c303 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:43.897247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243441ee-8659-45cb-9fae-3ec1b016c303 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:43.897538 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243441ee-8659-45cb-9fae-3ec1b016c303 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:43.897821 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-243441ee-8659-45cb-9fae-3ec1b016c303 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:43.898617 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243441ee-8659-45cb-9fae-3ec1b016c303 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:13:43.898926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243441ee-8659-45cb-9fae-3ec1b016c303 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:43.900533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-243441ee-8659-45cb-9fae-3ec1b016c303 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:43.913195 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1153/2319] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:13:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:13:43.917760 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244a934-7510-462f-8775-65836395be73 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:43.927399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9244a934-7510-462f-8775-65836395be73 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:43.932095 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6929022-4813-4975-8ac7-a6de01720048 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:43.932755 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6929022-4813-4975-8ac7-a6de01720048 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:43.933043 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6929022-4813-4975-8ac7-a6de01720048 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:43.933323 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6929022-4813-4975-8ac7-a6de01720048 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:43.934050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6929022-4813-4975-8ac7-a6de01720048 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:43.936026 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6929022-4813-4975-8ac7-a6de01720048 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:43.944272 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:13:43.965218 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1167/2320] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:13:43 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:13:43.969957 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243441ee-8659-45cb-9fae-3ec1b016c303 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:43.980479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-243441ee-8659-45cb-9fae-3ec1b016c303 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:44.039304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a64c6-58fe-499c-9815-5704570436e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:44.039980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a64c6-58fe-499c-9815-5704570436e3 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:44.040268 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a64c6-58fe-499c-9815-5704570436e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:44.040547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f76a64c6-58fe-499c-9815-5704570436e3 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:44.041245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a64c6-58fe-499c-9815-5704570436e3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:44.043209 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f76a64c6-58fe-499c-9815-5704570436e3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:44.064593 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1154/2321] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:13:43 2022] POST /identity/v3/users => generated 287 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:13:44.068943 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6929022-4813-4975-8ac7-a6de01720048 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:44.079541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6929022-4813-4975-8ac7-a6de01720048 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:44.084273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585c7d9-b1cd-466a-907b-514741311a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:44.084961 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585c7d9-b1cd-466a-907b-514741311a9c admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:44.085250 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585c7d9-b1cd-466a-907b-514741311a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:44.085525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9585c7d9-b1cd-466a-907b-514741311a9c admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:44.086251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585c7d9-b1cd-466a-907b-514741311a9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:44.087857 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9585c7d9-b1cd-466a-907b-514741311a9c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:44.098579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1168/2322] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:13:44.103644 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a64c6-58fe-499c-9815-5704570436e3 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:44.114395 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f76a64c6-58fe-499c-9815-5704570436e3 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:44.119081 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c9f0eca-1bd0-491d-b853-be4affc049c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:44.119738 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f0eca-1bd0-491d-b853-be4affc049c4 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:44.120029 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f0eca-1bd0-491d-b853-be4affc049c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:44.120311 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c9f0eca-1bd0-491d-b853-be4affc049c4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:44.121081 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f0eca-1bd0-491d-b853-be4affc049c4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:44.122701 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c9f0eca-1bd0-491d-b853-be4affc049c4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:44.132762 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1155/2323] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:13:44 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:13:44.136898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585c7d9-b1cd-466a-907b-514741311a9c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:44.147354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9585c7d9-b1cd-466a-907b-514741311a9c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:44.151986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28960cba-3c3d-4b77-b654-72d5d6d79cb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:44.152883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28960cba-3c3d-4b77-b654-72d5d6d79cb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:44.153164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28960cba-3c3d-4b77-b654-72d5d6d79cb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:44.153935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28960cba-3c3d-4b77-b654-72d5d6d79cb4 admin admin] PATH_INFO: `/v3/projects/a704e02f36bc4ae7b4c084d2abff0e0f/users/2e4813be17314acf84fe0312bc3628c6/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:44.168640 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28960cba-3c3d-4b77-b654-72d5d6d79cb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a704e02f36bc4ae7b4c084d2abff0e0f, user_id=2e4813be17314acf84fe0312bc3628c6, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:44.171849 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28960cba-3c3d-4b77-b654-72d5d6d79cb4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:44.185277 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1169/2324] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:13:44 2022] PUT /identity/v3/projects/a704e02f36bc4ae7b4c084d2abff0e0f/users/2e4813be17314acf84fe0312bc3628c6/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:13:44.195733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6e642-9aee-466c-9541-f4e3edda427d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:44.196035 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6e642-9aee-466c-9541-f4e3edda427d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:44.196341 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e6e642-9aee-466c-9541-f4e3edda427d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:44.222688 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e4e6e642-9aee-466c-9541-f4e3edda427d None None] MFA Rules not processed for user `2e4813be17314acf84fe0312bc3628c6`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:13:44.223935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e4e6e642-9aee-466c-9541-f4e3edda427d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:44.264649 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:13:44.291627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1156/2325] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:13:44 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:13:44.303576 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28960cba-3c3d-4b77-b654-72d5d6d79cb4 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:44.305300 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-28960cba-3c3d-4b77-b654-72d5d6d79cb4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:44.383599 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28960cba-3c3d-4b77-b654-72d5d6d79cb4 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:44.389207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53c4815-d7e9-4300-a0e7-1654190fe5aa service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:44.389883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c4815-d7e9-4300-a0e7-1654190fe5aa service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:44.390146 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c4815-d7e9-4300-a0e7-1654190fe5aa service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:44.390468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53c4815-d7e9-4300-a0e7-1654190fe5aa service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:44.398262 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c4815-d7e9-4300-a0e7-1654190fe5aa service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:44.399875 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53c4815-d7e9-4300-a0e7-1654190fe5aa service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:44.409135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1170/2326] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:13:44 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:13:47.067275 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e6e642-9aee-466c-9541-f4e3edda427d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:47.129325 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e6e642-9aee-466c-9541-f4e3edda427d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:47.133979 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3989b0-bd0e-4bbf-bb17-a7661facbf2d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:47.134878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3989b0-bd0e-4bbf-bb17-a7661facbf2d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:47.135171 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3989b0-bd0e-4bbf-bb17-a7661facbf2d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:47.135451 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3989b0-bd0e-4bbf-bb17-a7661facbf2d admin admin] PATH_INFO: `/v3/users/2e4813be17314acf84fe0312bc3628c6` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:47.136642 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3989b0-bd0e-4bbf-bb17-a7661facbf2d admin admin] RBAC: Authorizing `identity:delete_user(user_id=2e4813be17314acf84fe0312bc3628c6)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:47.138600 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f3989b0-bd0e-4bbf-bb17-a7661facbf2d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:47.185853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3f3989b0-bd0e-4bbf-bb17-a7661facbf2d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2e4813be17314acf84fe0312bc3628c6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:47.186540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3f3989b0-bd0e-4bbf-bb17-a7661facbf2d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2e4813be17314acf84fe0312bc3628c6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:47.194579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3f3989b0-bd0e-4bbf-bb17-a7661facbf2d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2e4813be17314acf84fe0312bc3628c6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:47.195204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3f3989b0-bd0e-4bbf-bb17-a7661facbf2d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2e4813be17314acf84fe0312bc3628c6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:47.207942 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3f3989b0-bd0e-4bbf-bb17-a7661facbf2d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2e4813be17314acf84fe0312bc3628c6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:47.221086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1157/2327] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:13:47 2022] DELETE /identity/v3/users/2e4813be17314acf84fe0312bc3628c6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:13:47.225667 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530f8a3-cf60-4979-b079-bdb5c3eeccf7 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:47.226286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530f8a3-cf60-4979-b079-bdb5c3eeccf7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:47.226286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5530f8a3-cf60-4979-b079-bdb5c3eeccf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:47.245060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5530f8a3-cf60-4979-b079-bdb5c3eeccf7 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:13:47.246282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5530f8a3-cf60-4979-b079-bdb5c3eeccf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:47.308032 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1171/2328] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:13:47.317712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3989b0-bd0e-4bbf-bb17-a7661facbf2d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:47.319246 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3f3989b0-bd0e-4bbf-bb17-a7661facbf2d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:47.377716 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f3989b0-bd0e-4bbf-bb17-a7661facbf2d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:47.381430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb64bfc4-860c-4454-9e2b-a877f73c63e6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:47.382079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64bfc4-860c-4454-9e2b-a877f73c63e6 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:47.382401 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64bfc4-860c-4454-9e2b-a877f73c63e6 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:47.382684 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb64bfc4-860c-4454-9e2b-a877f73c63e6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:47.391867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64bfc4-860c-4454-9e2b-a877f73c63e6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:47.393297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb64bfc4-860c-4454-9e2b-a877f73c63e6 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:47.402770 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1158/2329] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:13:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:13:47.672398 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530f8a3-cf60-4979-b079-bdb5c3eeccf7 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:47.673830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5530f8a3-cf60-4979-b079-bdb5c3eeccf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:47.688754 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5530f8a3-cf60-4979-b079-bdb5c3eeccf7 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:47.694499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f07b4131-e11b-4ded-a432-700a004dc53d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:47.695384 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b4131-e11b-4ded-a432-700a004dc53d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:47.695671 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b4131-e11b-4ded-a432-700a004dc53d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:47.695972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f07b4131-e11b-4ded-a432-700a004dc53d admin admin] PATH_INFO: `/v3/projects/a704e02f36bc4ae7b4c084d2abff0e0f` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:47.697169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b4131-e11b-4ded-a432-700a004dc53d admin admin] RBAC: Authorizing `identity:delete_project(project_id=a704e02f36bc4ae7b4c084d2abff0e0f)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:47.700043 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f07b4131-e11b-4ded-a432-700a004dc53d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:47.740805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:13:47.752585 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f07b4131-e11b-4ded-a432-700a004dc53d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a704e02f36bc4ae7b4c084d2abff0e0f', 'request_id': 'req-f07b4131-e11b-4ded-a432-700a004dc53d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:47.759998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f07b4131-e11b-4ded-a432-700a004dc53d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a704e02f36bc4ae7b4c084d2abff0e0f', 'request_id': 'req-f07b4131-e11b-4ded-a432-700a004dc53d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:47.765638 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f07b4131-e11b-4ded-a432-700a004dc53d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a704e02f36bc4ae7b4c084d2abff0e0f', 'request_id': 'req-f07b4131-e11b-4ded-a432-700a004dc53d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:47.776685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f07b4131-e11b-4ded-a432-700a004dc53d admin admin] The token cache is being invalidate because project a704e02f36bc4ae7b4c084d2abff0e0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:13:47.777002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f07b4131-e11b-4ded-a432-700a004dc53d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:47.777633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f07b4131-e11b-4ded-a432-700a004dc53d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:47.779244 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1172/2330] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:13:47 2022] DELETE /identity/v3/projects/a704e02f36bc4ae7b4c084d2abff0e0f => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:13:47.784227 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32092791-0302-4d3b-b995-f2fd395da22e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:47.784514 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32092791-0302-4d3b-b995-f2fd395da22e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:47.784808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32092791-0302-4d3b-b995-f2fd395da22e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:47.791520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:13:47.806029 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-32092791-0302-4d3b-b995-f2fd395da22e None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:13:47.807293 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-32092791-0302-4d3b-b995-f2fd395da22e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:47.871398 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1159/2331] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:13:47.880166 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b681716-c6e5-442d-920e-04cfa121b323 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:47.880462 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b681716-c6e5-442d-920e-04cfa121b323 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:47.880742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b681716-c6e5-442d-920e-04cfa121b323 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:47.899281 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-8b681716-c6e5-442d-920e-04cfa121b323 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:13:47.900506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-8b681716-c6e5-442d-920e-04cfa121b323 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:47.909029 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1173/2332] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:13:47.913729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32092791-0302-4d3b-b995-f2fd395da22e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:47.924685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32092791-0302-4d3b-b995-f2fd395da22e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:47.929583 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b494d0-e232-43fc-9dac-dddb8eb99f5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:47.930242 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b494d0-e232-43fc-9dac-dddb8eb99f5a admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:47.930553 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b494d0-e232-43fc-9dac-dddb8eb99f5a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:47.930833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b494d0-e232-43fc-9dac-dddb8eb99f5a admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:47.931571 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b494d0-e232-43fc-9dac-dddb8eb99f5a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:13:47.931886 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b494d0-e232-43fc-9dac-dddb8eb99f5a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:47.933499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b494d0-e232-43fc-9dac-dddb8eb99f5a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:47.945018 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1160/2333] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:13:47.949350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b681716-c6e5-442d-920e-04cfa121b323 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:47.958604 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b681716-c6e5-442d-920e-04cfa121b323 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:47.963367 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339d904-f020-4255-918c-eb43d9274550 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:47.964023 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339d904-f020-4255-918c-eb43d9274550 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:47.964310 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339d904-f020-4255-918c-eb43d9274550 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:47.964610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6339d904-f020-4255-918c-eb43d9274550 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:47.965304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6339d904-f020-4255-918c-eb43d9274550 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:47.967744 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6339d904-f020-4255-918c-eb43d9274550 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:47.993741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1174/2334] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:13:47 2022] POST /identity/v3/projects => generated 405 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:13:47.997914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b494d0-e232-43fc-9dac-dddb8eb99f5a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:48.008000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b494d0-e232-43fc-9dac-dddb8eb99f5a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:48.065763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff3402-a7a2-4172-9379-b96d88a1d411 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:48.066473 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff3402-a7a2-4172-9379-b96d88a1d411 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:48.066763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff3402-a7a2-4172-9379-b96d88a1d411 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:48.067042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8ff3402-a7a2-4172-9379-b96d88a1d411 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:48.067749 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff3402-a7a2-4172-9379-b96d88a1d411 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:48.069676 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8ff3402-a7a2-4172-9379-b96d88a1d411 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:48.089749 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1161/2335] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:13:47 2022] POST /identity/v3/users => generated 303 bytes in 92 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:13:48.094286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339d904-f020-4255-918c-eb43d9274550 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:48.105103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6339d904-f020-4255-918c-eb43d9274550 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:48.110460 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95205ed-f181-4f90-b54b-9df169ebe706 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:48.111191 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95205ed-f181-4f90-b54b-9df169ebe706 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:48.111482 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95205ed-f181-4f90-b54b-9df169ebe706 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:48.111762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95205ed-f181-4f90-b54b-9df169ebe706 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:48.112498 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95205ed-f181-4f90-b54b-9df169ebe706 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:48.114107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95205ed-f181-4f90-b54b-9df169ebe706 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:48.125845 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1175/2336] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:13:48.130379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff3402-a7a2-4172-9379-b96d88a1d411 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:48.140532 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8ff3402-a7a2-4172-9379-b96d88a1d411 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:48.145325 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cfd3235-7ed3-4806-afeb-a98b43f0e317 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:48.145991 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd3235-7ed3-4806-afeb-a98b43f0e317 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:48.146281 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd3235-7ed3-4806-afeb-a98b43f0e317 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:48.146594 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cfd3235-7ed3-4806-afeb-a98b43f0e317 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:48.147332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd3235-7ed3-4806-afeb-a98b43f0e317 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:48.148924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cfd3235-7ed3-4806-afeb-a98b43f0e317 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:48.163924 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1162/2337] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:13:48.165329 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95205ed-f181-4f90-b54b-9df169ebe706 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:48.177063 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95205ed-f181-4f90-b54b-9df169ebe706 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:48.181697 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23848c22-2b51-4ddb-a9b0-0cc7afd8d313 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:48.182615 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23848c22-2b51-4ddb-a9b0-0cc7afd8d313 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:48.182913 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23848c22-2b51-4ddb-a9b0-0cc7afd8d313 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:48.183185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23848c22-2b51-4ddb-a9b0-0cc7afd8d313 admin admin] PATH_INFO: `/v3/projects/9b7564fd46fd4bfb9286c8cc3dc9004b/users/bae57e70420b494dbfef7110df50120c/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:48.200505 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23848c22-2b51-4ddb-a9b0-0cc7afd8d313 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9b7564fd46fd4bfb9286c8cc3dc9004b, user_id=bae57e70420b494dbfef7110df50120c, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:48.203262 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23848c22-2b51-4ddb-a9b0-0cc7afd8d313 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:48.217578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1176/2338] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:13:48 2022] PUT /identity/v3/projects/9b7564fd46fd4bfb9286c8cc3dc9004b/users/bae57e70420b494dbfef7110df50120c/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:13:48.222653 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522da213-653b-4b38-bc04-1f0b92c7c3b5 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:48.223162 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522da213-653b-4b38-bc04-1f0b92c7c3b5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:48.223470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-522da213-653b-4b38-bc04-1f0b92c7c3b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:48.241999 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-522da213-653b-4b38-bc04-1f0b92c7c3b5 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:13:48.243250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-522da213-653b-4b38-bc04-1f0b92c7c3b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:48.304477 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1163/2339] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:13:48.314100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23848c22-2b51-4ddb-a9b0-0cc7afd8d313 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:48.316046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-23848c22-2b51-4ddb-a9b0-0cc7afd8d313 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:48.382097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23848c22-2b51-4ddb-a9b0-0cc7afd8d313 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:48.385909 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a5de4-b732-48f1-add3-d717fd4fe180 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:48.386626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a5de4-b732-48f1-add3-d717fd4fe180 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:48.386898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a5de4-b732-48f1-add3-d717fd4fe180 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:48.387176 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8a5de4-b732-48f1-add3-d717fd4fe180 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:48.398141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a5de4-b732-48f1-add3-d717fd4fe180 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:48.399718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a8a5de4-b732-48f1-add3-d717fd4fe180 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:48.413634 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1177/2340] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:13:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:13:49.048469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68666ee-d1c3-4b77-8529-24a2cbf22e11 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:49.049498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68666ee-d1c3-4b77-8529-24a2cbf22e11 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:49.050113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b68666ee-d1c3-4b77-8529-24a2cbf22e11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:49.070774 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:13:49.080675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b68666ee-d1c3-4b77-8529-24a2cbf22e11 None None] MFA Rules not processed for user `bae57e70420b494dbfef7110df50120c`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:13:49.083256 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b68666ee-d1c3-4b77-8529-24a2cbf22e11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:49.150487 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1164/2341] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:13:49 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:13:49.155029 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a5de4-b732-48f1-add3-d717fd4fe180 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:49.165307 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8a5de4-b732-48f1-add3-d717fd4fe180 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:49.170498 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5ff47-4910-4391-8ef5-6eec5372a907 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:49.171177 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5ff47-4910-4391-8ef5-6eec5372a907 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:49.171471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5ff47-4910-4391-8ef5-6eec5372a907 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:49.171891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d5ff47-4910-4391-8ef5-6eec5372a907 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:49.172661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5ff47-4910-4391-8ef5-6eec5372a907 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:49.174671 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d5ff47-4910-4391-8ef5-6eec5372a907 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:49.207544 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1178/2342] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:13:49 2022] POST /identity/v3/projects => generated 403 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:13:49.208671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68666ee-d1c3-4b77-8529-24a2cbf22e11 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:49.220666 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b68666ee-d1c3-4b77-8529-24a2cbf22e11 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:49.277444 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7839ca9-764e-4ca7-a172-a2205c33be2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:49.278167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7839ca9-764e-4ca7-a172-a2205c33be2f admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:49.278490 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7839ca9-764e-4ca7-a172-a2205c33be2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:49.278766 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7839ca9-764e-4ca7-a172-a2205c33be2f admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:49.279446 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7839ca9-764e-4ca7-a172-a2205c33be2f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:49.281339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7839ca9-764e-4ca7-a172-a2205c33be2f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:49.300726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1165/2343] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:13:49 2022] POST /identity/v3/users => generated 308 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:13:49.304785 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5ff47-4910-4391-8ef5-6eec5372a907 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:49.315809 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d5ff47-4910-4391-8ef5-6eec5372a907 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:49.320433 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc0a90-ff07-4d24-80c3-524d11d4e1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:49.321081 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc0a90-ff07-4d24-80c3-524d11d4e1d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:49.321383 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc0a90-ff07-4d24-80c3-524d11d4e1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:49.321644 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0dc0a90-ff07-4d24-80c3-524d11d4e1d5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:49.322410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc0a90-ff07-4d24-80c3-524d11d4e1d5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:49.324007 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0dc0a90-ff07-4d24-80c3-524d11d4e1d5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:49.334528 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1179/2344] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:13:49.338709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7839ca9-764e-4ca7-a172-a2205c33be2f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:49.349397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7839ca9-764e-4ca7-a172-a2205c33be2f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:49.354029 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25604893-cd20-4381-ad66-7188e5a9b0f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:49.354931 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25604893-cd20-4381-ad66-7188e5a9b0f4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:49.355212 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25604893-cd20-4381-ad66-7188e5a9b0f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:49.355485 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25604893-cd20-4381-ad66-7188e5a9b0f4 admin admin] PATH_INFO: `/v3/projects/d0fbc91785184b88ad012b6ef3aae5ab/users/50f85934478044d79444a7207c3e0dbd/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:49.370807 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25604893-cd20-4381-ad66-7188e5a9b0f4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0fbc91785184b88ad012b6ef3aae5ab, user_id=50f85934478044d79444a7207c3e0dbd, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:49.373486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25604893-cd20-4381-ad66-7188e5a9b0f4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:49.386669 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1166/2345] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:13:49 2022] PUT /identity/v3/projects/d0fbc91785184b88ad012b6ef3aae5ab/users/50f85934478044d79444a7207c3e0dbd/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:13:49.390744 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc0a90-ff07-4d24-80c3-524d11d4e1d5 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:49.400765 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0dc0a90-ff07-4d24-80c3-524d11d4e1d5 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:49.427460 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=5,keystone:SELECT=91,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:13:49.459569 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3e6cc39-3566-4a07-82c5-5652fa3b2f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:49.460236 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6cc39-3566-4a07-82c5-5652fa3b2f79 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:49.460520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6cc39-3566-4a07-82c5-5652fa3b2f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:49.460798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3e6cc39-3566-4a07-82c5-5652fa3b2f79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:49.461528 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6cc39-3566-4a07-82c5-5652fa3b2f79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:49.463153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3e6cc39-3566-4a07-82c5-5652fa3b2f79 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:49.473861 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1180/2346] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:13:49 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:13:49.478228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25604893-cd20-4381-ad66-7188e5a9b0f4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:49.488753 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25604893-cd20-4381-ad66-7188e5a9b0f4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:49.493412 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7fd6d-24f8-434d-9aa6-66c9f37cac0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:49.494319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7fd6d-24f8-434d-9aa6-66c9f37cac0a admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:49.494622 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7fd6d-24f8-434d-9aa6-66c9f37cac0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:49.494903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd7fd6d-24f8-434d-9aa6-66c9f37cac0a admin admin] PATH_INFO: `/v3/projects/d0fbc91785184b88ad012b6ef3aae5ab/users/50f85934478044d79444a7207c3e0dbd/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:49.497471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7fd6d-24f8-434d-9aa6-66c9f37cac0a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d0fbc91785184b88ad012b6ef3aae5ab, user_id=50f85934478044d79444a7207c3e0dbd, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:49.500153 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd7fd6d-24f8-434d-9aa6-66c9f37cac0a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:49.510475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1167/2347] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:13:49 2022] PUT /identity/v3/projects/d0fbc91785184b88ad012b6ef3aae5ab/users/50f85934478044d79444a7207c3e0dbd/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:13:50.241329 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b64c3b-3ec1-4fd5-99b8-d4326a533ed9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:50.241627 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b64c3b-3ec1-4fd5-99b8-d4326a533ed9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:50.241911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b64c3b-3ec1-4fd5-99b8-d4326a533ed9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:50.271900 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-49b64c3b-3ec1-4fd5-99b8-d4326a533ed9 None None] MFA Rules not processed for user `50f85934478044d79444a7207c3e0dbd`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:13:50.273111 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-49b64c3b-3ec1-4fd5-99b8-d4326a533ed9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:50.361596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1181/2348] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:13:50 2022] POST /identity/v3/auth/tokens => generated 3337 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:13:51.560394 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7fd6d-24f8-434d-9aa6-66c9f37cac0a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:51.615258 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:13:51.629707 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd7fd6d-24f8-434d-9aa6-66c9f37cac0a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:51.634574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbc058-3e19-4c36-aa1c-cb6c129597ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:51.635458 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbc058-3e19-4c36-aa1c-cb6c129597ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:51.635807 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbc058-3e19-4c36-aa1c-cb6c129597ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:51.636009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22fbc058-3e19-4c36-aa1c-cb6c129597ef admin admin] PATH_INFO: `/v3/users/bae57e70420b494dbfef7110df50120c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:51.637217 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fbc058-3e19-4c36-aa1c-cb6c129597ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=bae57e70420b494dbfef7110df50120c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:51.639830 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22fbc058-3e19-4c36-aa1c-cb6c129597ef admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:51.690228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-22fbc058-3e19-4c36-aa1c-cb6c129597ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bae57e70420b494dbfef7110df50120c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:51.691007 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-22fbc058-3e19-4c36-aa1c-cb6c129597ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bae57e70420b494dbfef7110df50120c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:51.698823 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-22fbc058-3e19-4c36-aa1c-cb6c129597ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bae57e70420b494dbfef7110df50120c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:51.699487 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-22fbc058-3e19-4c36-aa1c-cb6c129597ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bae57e70420b494dbfef7110df50120c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:51.712619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-22fbc058-3e19-4c36-aa1c-cb6c129597ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bae57e70420b494dbfef7110df50120c'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:51.727542 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1168/2349] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:13:51 2022] DELETE /identity/v3/users/bae57e70420b494dbfef7110df50120c => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:13:51.927266 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b64c3b-3ec1-4fd5-99b8-d4326a533ed9 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:51.945597 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b64c3b-3ec1-4fd5-99b8-d4326a533ed9 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:52.030223 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43909a0a-d60c-4384-92d3-a78250abdd3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:52.031154 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43909a0a-d60c-4384-92d3-a78250abdd3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:52.031445 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43909a0a-d60c-4384-92d3-a78250abdd3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:52.031729 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43909a0a-d60c-4384-92d3-a78250abdd3d admin admin] PATH_INFO: `/v3/projects/9b7564fd46fd4bfb9286c8cc3dc9004b` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:52.033079 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43909a0a-d60c-4384-92d3-a78250abdd3d admin admin] RBAC: Authorizing `identity:delete_project(project_id=9b7564fd46fd4bfb9286c8cc3dc9004b)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:52.036980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43909a0a-d60c-4384-92d3-a78250abdd3d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:52.102546 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-43909a0a-d60c-4384-92d3-a78250abdd3d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9b7564fd46fd4bfb9286c8cc3dc9004b', 'request_id': 'req-43909a0a-d60c-4384-92d3-a78250abdd3d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:52.105055 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:13:52.112071 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-43909a0a-d60c-4384-92d3-a78250abdd3d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9b7564fd46fd4bfb9286c8cc3dc9004b', 'request_id': 'req-43909a0a-d60c-4384-92d3-a78250abdd3d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:52.119699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-43909a0a-d60c-4384-92d3-a78250abdd3d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9b7564fd46fd4bfb9286c8cc3dc9004b', 'request_id': 'req-43909a0a-d60c-4384-92d3-a78250abdd3d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:52.131022 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-43909a0a-d60c-4384-92d3-a78250abdd3d admin admin] The token cache is being invalidate because project 9b7564fd46fd4bfb9286c8cc3dc9004b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:13:52.131319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-43909a0a-d60c-4384-92d3-a78250abdd3d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:52.131973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-43909a0a-d60c-4384-92d3-a78250abdd3d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:52.133766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1182/2350] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:13:51 2022] DELETE /identity/v3/projects/9b7564fd46fd4bfb9286c8cc3dc9004b => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:13:52.138288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbc058-3e19-4c36-aa1c-cb6c129597ef admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:52.139821 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-22fbc058-3e19-4c36-aa1c-cb6c129597ef admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:52.153736 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22fbc058-3e19-4c36-aa1c-cb6c129597ef admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:52.225632 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fffe311-3375-4302-b8d2-dbc4f1df6fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:52.226622 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffe311-3375-4302-b8d2-dbc4f1df6fe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:52.226860 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffe311-3375-4302-b8d2-dbc4f1df6fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:52.227430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fffe311-3375-4302-b8d2-dbc4f1df6fe2 admin admin] PATH_INFO: `/v3/users/50f85934478044d79444a7207c3e0dbd` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:52.228480 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fffe311-3375-4302-b8d2-dbc4f1df6fe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=50f85934478044d79444a7207c3e0dbd)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:52.231698 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fffe311-3375-4302-b8d2-dbc4f1df6fe2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:52.282998 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9fffe311-3375-4302-b8d2-dbc4f1df6fe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '50f85934478044d79444a7207c3e0dbd'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:52.284111 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9fffe311-3375-4302-b8d2-dbc4f1df6fe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '50f85934478044d79444a7207c3e0dbd'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:52.291652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9fffe311-3375-4302-b8d2-dbc4f1df6fe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '50f85934478044d79444a7207c3e0dbd'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:52.292272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9fffe311-3375-4302-b8d2-dbc4f1df6fe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '50f85934478044d79444a7207c3e0dbd'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:52.305377 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-9fffe311-3375-4302-b8d2-dbc4f1df6fe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '50f85934478044d79444a7207c3e0dbd'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:52.306580 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:13:52.321451 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1169/2351] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:13:52 2022] DELETE /identity/v3/users/50f85934478044d79444a7207c3e0dbd => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:13:52.508029 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43909a0a-d60c-4384-92d3-a78250abdd3d admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:52.509954 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-43909a0a-d60c-4384-92d3-a78250abdd3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:52.538177 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43909a0a-d60c-4384-92d3-a78250abdd3d admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:52.598684 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f0713a-103a-467b-b971-a37f597102a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:52.599557 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0713a-103a-467b-b971-a37f597102a8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:52.599837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0713a-103a-467b-b971-a37f597102a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:52.600110 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f0713a-103a-467b-b971-a37f597102a8 admin admin] PATH_INFO: `/v3/projects/d0fbc91785184b88ad012b6ef3aae5ab` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:52.601333 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0713a-103a-467b-b971-a37f597102a8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d0fbc91785184b88ad012b6ef3aae5ab)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:52.603307 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f0713a-103a-467b-b971-a37f597102a8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:52.656960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c9f0713a-103a-467b-b971-a37f597102a8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd0fbc91785184b88ad012b6ef3aae5ab', 'request_id': 'req-c9f0713a-103a-467b-b971-a37f597102a8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:52.665478 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c9f0713a-103a-467b-b971-a37f597102a8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd0fbc91785184b88ad012b6ef3aae5ab', 'request_id': 'req-c9f0713a-103a-467b-b971-a37f597102a8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:52.672383 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c9f0713a-103a-467b-b971-a37f597102a8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd0fbc91785184b88ad012b6ef3aae5ab', 'request_id': 'req-c9f0713a-103a-467b-b971-a37f597102a8'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:52.687259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c9f0713a-103a-467b-b971-a37f597102a8 admin admin] The token cache is being invalidate because project d0fbc91785184b88ad012b6ef3aae5ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:13:52.687550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c9f0713a-103a-467b-b971-a37f597102a8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:52.688378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c9f0713a-103a-467b-b971-a37f597102a8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:13:52.690174 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1183/2352] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:13:52 2022] DELETE /identity/v3/projects/d0fbc91785184b88ad012b6ef3aae5ab => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:13:52.696332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b842826-954c-461a-a112-c86cbb306196 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:52.698053 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b842826-954c-461a-a112-c86cbb306196 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:52.698581 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b842826-954c-461a-a112-c86cbb306196 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:52.730126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-4b842826-954c-461a-a112-c86cbb306196 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:13:52.732645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4b842826-954c-461a-a112-c86cbb306196 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:52.820075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1170/2353] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:13:52.829562 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cee6b-a2a5-43f4-9b52-7448961c61c6 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:52.830007 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cee6b-a2a5-43f4-9b52-7448961c61c6 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:52.830321 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9cee6b-a2a5-43f4-9b52-7448961c61c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:52.851659 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-6a9cee6b-a2a5-43f4-9b52-7448961c61c6 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:13:52.852881 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6a9cee6b-a2a5-43f4-9b52-7448961c61c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:52.861397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1184/2354] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:13:52.865751 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b842826-954c-461a-a112-c86cbb306196 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:52.876361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b842826-954c-461a-a112-c86cbb306196 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:52.881307 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35197ed-f341-48c2-9bbf-e38628c016a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:52.882099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35197ed-f341-48c2-9bbf-e38628c016a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:52.882426 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35197ed-f341-48c2-9bbf-e38628c016a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:52.884755 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f35197ed-f341-48c2-9bbf-e38628c016a0 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:52.884755 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35197ed-f341-48c2-9bbf-e38628c016a0 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:13:52.884755 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35197ed-f341-48c2-9bbf-e38628c016a0 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:52.885690 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f35197ed-f341-48c2-9bbf-e38628c016a0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:52.900286 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1171/2355] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:13:52.905265 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9cee6b-a2a5-43f4-9b52-7448961c61c6 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:52.915458 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9cee6b-a2a5-43f4-9b52-7448961c61c6 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:52.921110 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21ec76-ad7f-4c99-b926-619bdd39fab1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:52.922038 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21ec76-ad7f-4c99-b926-619bdd39fab1 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:52.922377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21ec76-ad7f-4c99-b926-619bdd39fab1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:52.922750 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a21ec76-ad7f-4c99-b926-619bdd39fab1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:52.923494 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21ec76-ad7f-4c99-b926-619bdd39fab1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:52.925848 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a21ec76-ad7f-4c99-b926-619bdd39fab1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:52.958155 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1185/2356] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:13:52 2022] POST /identity/v3/projects => generated 371 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:13:52.962218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35197ed-f341-48c2-9bbf-e38628c016a0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:52.974020 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f35197ed-f341-48c2-9bbf-e38628c016a0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:53.037116 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95bb75-e5ae-4723-a249-5d708c98a154 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:53.037724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95bb75-e5ae-4723-a249-5d708c98a154 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:53.038043 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95bb75-e5ae-4723-a249-5d708c98a154 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:53.038326 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95bb75-e5ae-4723-a249-5d708c98a154 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:53.039273 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95bb75-e5ae-4723-a249-5d708c98a154 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:53.042105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be95bb75-e5ae-4723-a249-5d708c98a154 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:53.069393 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1172/2357] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:13:52 2022] POST /identity/v3/users => generated 286 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:13:53.070762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21ec76-ad7f-4c99-b926-619bdd39fab1 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:53.083650 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a21ec76-ad7f-4c99-b926-619bdd39fab1 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:53.089732 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd32cc-267a-4900-9c8c-a351c03e66a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:53.090971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd32cc-267a-4900-9c8c-a351c03e66a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:53.091265 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd32cc-267a-4900-9c8c-a351c03e66a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:53.091547 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cd32cc-267a-4900-9c8c-a351c03e66a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:53.092435 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd32cc-267a-4900-9c8c-a351c03e66a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:53.094069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70cd32cc-267a-4900-9c8c-a351c03e66a5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:53.105688 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1186/2358] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:13:53.111334 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95bb75-e5ae-4723-a249-5d708c98a154 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:53.122434 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95bb75-e5ae-4723-a249-5d708c98a154 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:53.128078 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10a21ebe-2a7a-45ac-b90e-70fcf1ec762f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:53.128846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a21ebe-2a7a-45ac-b90e-70fcf1ec762f admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:53.129195 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a21ebe-2a7a-45ac-b90e-70fcf1ec762f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:53.129543 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10a21ebe-2a7a-45ac-b90e-70fcf1ec762f admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:53.130378 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a21ebe-2a7a-45ac-b90e-70fcf1ec762f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:53.132137 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10a21ebe-2a7a-45ac-b90e-70fcf1ec762f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:53.143704 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1173/2359] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:13:53.150068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd32cc-267a-4900-9c8c-a351c03e66a5 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:53.161495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cd32cc-267a-4900-9c8c-a351c03e66a5 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:53.166961 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad81cb-289e-4824-83f8-176768e363b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:53.167875 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad81cb-289e-4824-83f8-176768e363b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:53.168159 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad81cb-289e-4824-83f8-176768e363b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:53.168440 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaad81cb-289e-4824-83f8-176768e363b2 admin admin] PATH_INFO: `/v3/projects/ace883e3994944d18a5a7ecae60cdd79/users/d405887bc4d54f37b9e3693ce6f2e3ff/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:53.187706 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad81cb-289e-4824-83f8-176768e363b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ace883e3994944d18a5a7ecae60cdd79, user_id=d405887bc4d54f37b9e3693ce6f2e3ff, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:53.190484 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaad81cb-289e-4824-83f8-176768e363b2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:53.204789 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1187/2360] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:13:53 2022] PUT /identity/v3/projects/ace883e3994944d18a5a7ecae60cdd79/users/d405887bc4d54f37b9e3693ce6f2e3ff/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:13:53.217546 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeff11b-22fa-4e02-bad0-65c1b4ec62b7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:53.218603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeff11b-22fa-4e02-bad0-65c1b4ec62b7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:53.218940 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebeff11b-22fa-4e02-bad0-65c1b4ec62b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:53.251029 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ebeff11b-22fa-4e02-bad0-65c1b4ec62b7 None None] MFA Rules not processed for user `d405887bc4d54f37b9e3693ce6f2e3ff`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:13:53.252343 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ebeff11b-22fa-4e02-bad0-65c1b4ec62b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:53.300954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:13:53.322823 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1174/2361] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:13:53 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:13:53.327672 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad81cb-289e-4824-83f8-176768e363b2 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:53.337828 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=10,keystone:SELECT=87,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:13:53.398439 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaad81cb-289e-4824-83f8-176768e363b2 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:53.403793 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92ee8e-fb97-4ab6-a8f4-9ad3353c5abb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:53.404666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92ee8e-fb97-4ab6-a8f4-9ad3353c5abb admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:53.404958 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92ee8e-fb97-4ab6-a8f4-9ad3353c5abb admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:53.405238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b92ee8e-fb97-4ab6-a8f4-9ad3353c5abb admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:53.405943 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92ee8e-fb97-4ab6-a8f4-9ad3353c5abb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:53.407916 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b92ee8e-fb97-4ab6-a8f4-9ad3353c5abb admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:53.434549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1188/2362] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:13:53 2022] POST /identity/v3/projects => generated 371 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:13:53.438952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeff11b-22fa-4e02-bad0-65c1b4ec62b7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:53.450753 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebeff11b-22fa-4e02-bad0-65c1b4ec62b7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:53.508133 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4aa3d-9d65-4aa5-ab57-c6c5a46af222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:53.508816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4aa3d-9d65-4aa5-ab57-c6c5a46af222 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:53.509100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4aa3d-9d65-4aa5-ab57-c6c5a46af222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:53.509378 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0f4aa3d-9d65-4aa5-ab57-c6c5a46af222 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:53.510097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4aa3d-9d65-4aa5-ab57-c6c5a46af222 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:53.512026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0f4aa3d-9d65-4aa5-ab57-c6c5a46af222 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:53.536585 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1175/2363] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:13:53 2022] POST /identity/v3/users => generated 292 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:13:53.537746 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92ee8e-fb97-4ab6-a8f4-9ad3353c5abb admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:53.548356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b92ee8e-fb97-4ab6-a8f4-9ad3353c5abb admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:53.553739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca79074-27a9-4266-8e74-9b3e33c48dce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:53.554465 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca79074-27a9-4266-8e74-9b3e33c48dce admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:53.554842 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca79074-27a9-4266-8e74-9b3e33c48dce admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:53.555117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca79074-27a9-4266-8e74-9b3e33c48dce admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:53.555858 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca79074-27a9-4266-8e74-9b3e33c48dce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:53.558045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca79074-27a9-4266-8e74-9b3e33c48dce admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:53.572956 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1189/2364] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:13:53.573794 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4aa3d-9d65-4aa5-ab57-c6c5a46af222 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:53.584425 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0f4aa3d-9d65-4aa5-ab57-c6c5a46af222 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:53.589239 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39dd105-44c0-490a-8011-84324ef26d05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:53.590125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39dd105-44c0-490a-8011-84324ef26d05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:53.591035 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39dd105-44c0-490a-8011-84324ef26d05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:53.591333 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e39dd105-44c0-490a-8011-84324ef26d05 admin admin] PATH_INFO: `/v3/projects/09b0437472484bb4ac61f95e5f4c3ebf/users/784c894fecd347d1b0a14f84e151eb20/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:53.608363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39dd105-44c0-490a-8011-84324ef26d05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b0437472484bb4ac61f95e5f4c3ebf, user_id=784c894fecd347d1b0a14f84e151eb20, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:53.611113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e39dd105-44c0-490a-8011-84324ef26d05 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:53.624968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1176/2365] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:13:53 2022] PUT /identity/v3/projects/09b0437472484bb4ac61f95e5f4c3ebf/users/784c894fecd347d1b0a14f84e151eb20/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:13:53.628990 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca79074-27a9-4266-8e74-9b3e33c48dce admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:53.641011 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca79074-27a9-4266-8e74-9b3e33c48dce admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:53.702710 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cda8fa5-65e8-4a37-bb99-0e897ef90aba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:53.703391 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda8fa5-65e8-4a37-bb99-0e897ef90aba admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:53.703680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda8fa5-65e8-4a37-bb99-0e897ef90aba admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:53.703961 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cda8fa5-65e8-4a37-bb99-0e897ef90aba admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:53.704699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda8fa5-65e8-4a37-bb99-0e897ef90aba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:53.706347 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cda8fa5-65e8-4a37-bb99-0e897ef90aba admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:53.720124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1190/2366] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:13:53 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:13:53.721407 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39dd105-44c0-490a-8011-84324ef26d05 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:53.732576 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e39dd105-44c0-490a-8011-84324ef26d05 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:53.737612 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774ec5f-9731-4c9e-a90f-c710a81fc8be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:53.738530 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774ec5f-9731-4c9e-a90f-c710a81fc8be admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:53.738810 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774ec5f-9731-4c9e-a90f-c710a81fc8be admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:53.739085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f774ec5f-9731-4c9e-a90f-c710a81fc8be admin admin] PATH_INFO: `/v3/projects/09b0437472484bb4ac61f95e5f4c3ebf/users/784c894fecd347d1b0a14f84e151eb20/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:53.741109 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774ec5f-9731-4c9e-a90f-c710a81fc8be admin admin] RBAC: Authorizing `identity:create_grant(project_id=09b0437472484bb4ac61f95e5f4c3ebf, user_id=784c894fecd347d1b0a14f84e151eb20, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:53.743907 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f774ec5f-9731-4c9e-a90f-c710a81fc8be admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:53.756020 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1177/2367] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:13:53 2022] PUT /identity/v3/projects/09b0437472484bb4ac61f95e5f4c3ebf/users/784c894fecd347d1b0a14f84e151eb20/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:13:53.767083 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec930dd-8c8b-45ab-bbf8-dffeea4b4ec0 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:53.767376 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec930dd-8c8b-45ab-bbf8-dffeea4b4ec0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:53.767658 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cec930dd-8c8b-45ab-bbf8-dffeea4b4ec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:53.796642 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-cec930dd-8c8b-45ab-bbf8-dffeea4b4ec0 None None] MFA Rules not processed for user `784c894fecd347d1b0a14f84e151eb20`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:13:53.797879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-cec930dd-8c8b-45ab-bbf8-dffeea4b4ec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:53.870665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1191/2368] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:13:53 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:13:53.881832 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774ec5f-9731-4c9e-a90f-c710a81fc8be admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:53.883658 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f774ec5f-9731-4c9e-a90f-c710a81fc8be admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:53.945989 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f774ec5f-9731-4c9e-a90f-c710a81fc8be admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:53.949819 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14742fea-a260-4c88-9c05-baa2dfd8987d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:53.950532 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14742fea-a260-4c88-9c05-baa2dfd8987d service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:53.950798 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14742fea-a260-4c88-9c05-baa2dfd8987d service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:53.951066 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14742fea-a260-4c88-9c05-baa2dfd8987d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:53.960753 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14742fea-a260-4c88-9c05-baa2dfd8987d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:53.962207 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14742fea-a260-4c88-9c05-baa2dfd8987d service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:53.972372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1178/2369] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:13:53 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:13:54.056453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd13589-e6e4-4ee7-b388-1c073181148a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:54.056956 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd13589-e6e4-4ee7-b388-1c073181148a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:54.057061 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd13589-e6e4-4ee7-b388-1c073181148a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:54.063587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:13:54.063763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14742fea-a260-4c88-9c05-baa2dfd8987d service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:13:54.072720 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14742fea-a260-4c88-9c05-baa2dfd8987d service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:13:54.076621 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58f6ca3d-6ec5-4e85-90a1-288aa07d1fc4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:13:54.077276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6ca3d-6ec5-4e85-90a1-288aa07d1fc4 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:13:54.077961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6ca3d-6ec5-4e85-90a1-288aa07d1fc4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:13:54.077961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58f6ca3d-6ec5-4e85-90a1-288aa07d1fc4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:13:54.082694 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4dd13589-e6e4-4ee7-b388-1c073181148a None None] MFA Rules not processed for user `296853a88430438e96e0f8fb97b46d2e`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:13:54.083882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4dd13589-e6e4-4ee7-b388-1c073181148a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:13:54.122964 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:13:54.142243 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1192/2370] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:13:54 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:13:54.143611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6ca3d-6ec5-4e85-90a1-288aa07d1fc4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:13:54.145058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58f6ca3d-6ec5-4e85-90a1-288aa07d1fc4 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:13:54.155491 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1179/2371] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:13:54 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:14:04.134442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=24 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:14:04.150547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=10 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:14:39.814610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd13589-e6e4-4ee7-b388-1c073181148a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:14:39.882514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd13589-e6e4-4ee7-b388-1c073181148a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:14:39.888250 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca48ac-390e-4992-89b7-6125d61a640b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:14:39.889149 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca48ac-390e-4992-89b7-6125d61a640b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:39.889428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca48ac-390e-4992-89b7-6125d61a640b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:39.889701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4ca48ac-390e-4992-89b7-6125d61a640b admin admin] PATH_INFO: `/v3/users/d405887bc4d54f37b9e3693ce6f2e3ff` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:39.890928 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ca48ac-390e-4992-89b7-6125d61a640b admin admin] RBAC: Authorizing `identity:delete_user(user_id=d405887bc4d54f37b9e3693ce6f2e3ff)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:14:39.892913 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4ca48ac-390e-4992-89b7-6125d61a640b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:14:39.944978 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f4ca48ac-390e-4992-89b7-6125d61a640b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd405887bc4d54f37b9e3693ce6f2e3ff'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:39.957601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f4ca48ac-390e-4992-89b7-6125d61a640b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd405887bc4d54f37b9e3693ce6f2e3ff'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:39.967825 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f4ca48ac-390e-4992-89b7-6125d61a640b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd405887bc4d54f37b9e3693ce6f2e3ff'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:39.968780 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f4ca48ac-390e-4992-89b7-6125d61a640b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd405887bc4d54f37b9e3693ce6f2e3ff'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:39.978440 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f4ca48ac-390e-4992-89b7-6125d61a640b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd405887bc4d54f37b9e3693ce6f2e3ff'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:39.986495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1193/2372] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:14:39 2022] DELETE /identity/v3/users/d405887bc4d54f37b9e3693ce6f2e3ff => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:14:39.991398 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c5e59-a473-422a-ad22-7d8335c2010f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:39.992178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c5e59-a473-422a-ad22-7d8335c2010f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:39.992178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4c5e59-a473-422a-ad22-7d8335c2010f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:40.011076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-fe4c5e59-a473-422a-ad22-7d8335c2010f None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:14:40.012328 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fe4c5e59-a473-422a-ad22-7d8335c2010f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:14:40.076555 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1180/2373] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:14:40.086272 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca48ac-390e-4992-89b7-6125d61a640b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:14:40.087712 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f4ca48ac-390e-4992-89b7-6125d61a640b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:14:40.152591 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4ca48ac-390e-4992-89b7-6125d61a640b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:14:40.156365 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf578e-dfe0-4184-a72d-62363e1cbedb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:14:40.157020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf578e-dfe0-4184-a72d-62363e1cbedb service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:40.157303 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf578e-dfe0-4184-a72d-62363e1cbedb service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:40.157577 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cf578e-dfe0-4184-a72d-62363e1cbedb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:40.167247 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf578e-dfe0-4184-a72d-62363e1cbedb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:14:40.169212 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cf578e-dfe0-4184-a72d-62363e1cbedb service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:14:40.179514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1194/2374] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:14:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:14:40.459933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c5e59-a473-422a-ad22-7d8335c2010f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:14:40.461427 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fe4c5e59-a473-422a-ad22-7d8335c2010f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:14:40.474584 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4c5e59-a473-422a-ad22-7d8335c2010f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:14:40.481486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf367edb-156f-406b-9416-0e2a55f493b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:14:40.482463 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf367edb-156f-406b-9416-0e2a55f493b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:40.482753 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf367edb-156f-406b-9416-0e2a55f493b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:40.483055 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf367edb-156f-406b-9416-0e2a55f493b2 admin admin] PATH_INFO: `/v3/projects/ace883e3994944d18a5a7ecae60cdd79` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:40.484566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf367edb-156f-406b-9416-0e2a55f493b2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ace883e3994944d18a5a7ecae60cdd79)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:14:40.486532 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf367edb-156f-406b-9416-0e2a55f493b2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:14:40.538437 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cf367edb-156f-406b-9416-0e2a55f493b2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ace883e3994944d18a5a7ecae60cdd79', 'request_id': 'req-cf367edb-156f-406b-9416-0e2a55f493b2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:40.546671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cf367edb-156f-406b-9416-0e2a55f493b2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ace883e3994944d18a5a7ecae60cdd79', 'request_id': 'req-cf367edb-156f-406b-9416-0e2a55f493b2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:40.555238 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cf367edb-156f-406b-9416-0e2a55f493b2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ace883e3994944d18a5a7ecae60cdd79', 'request_id': 'req-cf367edb-156f-406b-9416-0e2a55f493b2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:40.567293 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cf367edb-156f-406b-9416-0e2a55f493b2 admin admin] The token cache is being invalidate because project ace883e3994944d18a5a7ecae60cdd79 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:14:40.567637 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cf367edb-156f-406b-9416-0e2a55f493b2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:40.568428 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cf367edb-156f-406b-9416-0e2a55f493b2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:40.570150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1181/2375] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:14:40 2022] DELETE /identity/v3/projects/ace883e3994944d18a5a7ecae60cdd79 => generated 0 bytes in 111 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:14:40.574714 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf578e-dfe0-4184-a72d-62363e1cbedb service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:14:40.584742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cf578e-dfe0-4184-a72d-62363e1cbedb service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:14:40.604950 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:14:40.648600 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e5bb50-2c53-4c3b-b451-e6d12d9f6228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:14:40.649553 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5bb50-2c53-4c3b-b451-e6d12d9f6228 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:40.649859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5bb50-2c53-4c3b-b451-e6d12d9f6228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:40.650140 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e5bb50-2c53-4c3b-b451-e6d12d9f6228 admin admin] PATH_INFO: `/v3/users/784c894fecd347d1b0a14f84e151eb20` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:40.651476 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5bb50-2c53-4c3b-b451-e6d12d9f6228 admin admin] RBAC: Authorizing `identity:delete_user(user_id=784c894fecd347d1b0a14f84e151eb20)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:14:40.654048 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e5bb50-2c53-4c3b-b451-e6d12d9f6228 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:14:40.703067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-73e5bb50-2c53-4c3b-b451-e6d12d9f6228 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '784c894fecd347d1b0a14f84e151eb20'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:40.726239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-73e5bb50-2c53-4c3b-b451-e6d12d9f6228 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '784c894fecd347d1b0a14f84e151eb20'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:40.735029 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-73e5bb50-2c53-4c3b-b451-e6d12d9f6228 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '784c894fecd347d1b0a14f84e151eb20'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:40.735681 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-73e5bb50-2c53-4c3b-b451-e6d12d9f6228 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '784c894fecd347d1b0a14f84e151eb20'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:40.744175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-73e5bb50-2c53-4c3b-b451-e6d12d9f6228 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '784c894fecd347d1b0a14f84e151eb20'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:40.751216 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1195/2376] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:14:40 2022] DELETE /identity/v3/users/784c894fecd347d1b0a14f84e151eb20 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:14:41.060012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf367edb-156f-406b-9416-0e2a55f493b2 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:14:41.061533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cf367edb-156f-406b-9416-0e2a55f493b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:14:41.083931 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf367edb-156f-406b-9416-0e2a55f493b2 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:14:41.142522 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d31f36ef-30ea-4e78-a92b-8b30b6993442 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:14:41.143404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f36ef-30ea-4e78-a92b-8b30b6993442 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:41.143715 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f36ef-30ea-4e78-a92b-8b30b6993442 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:41.144010 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d31f36ef-30ea-4e78-a92b-8b30b6993442 admin admin] PATH_INFO: `/v3/projects/09b0437472484bb4ac61f95e5f4c3ebf` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:41.145219 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f36ef-30ea-4e78-a92b-8b30b6993442 admin admin] RBAC: Authorizing `identity:delete_project(project_id=09b0437472484bb4ac61f95e5f4c3ebf)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:14:41.147164 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d31f36ef-30ea-4e78-a92b-8b30b6993442 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:14:41.178259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:14:41.198046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d31f36ef-30ea-4e78-a92b-8b30b6993442 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '09b0437472484bb4ac61f95e5f4c3ebf', 'request_id': 'req-d31f36ef-30ea-4e78-a92b-8b30b6993442'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:41.204337 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d31f36ef-30ea-4e78-a92b-8b30b6993442 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '09b0437472484bb4ac61f95e5f4c3ebf', 'request_id': 'req-d31f36ef-30ea-4e78-a92b-8b30b6993442'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:41.211816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d31f36ef-30ea-4e78-a92b-8b30b6993442 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '09b0437472484bb4ac61f95e5f4c3ebf', 'request_id': 'req-d31f36ef-30ea-4e78-a92b-8b30b6993442'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:41.223479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d31f36ef-30ea-4e78-a92b-8b30b6993442 admin admin] The token cache is being invalidate because project 09b0437472484bb4ac61f95e5f4c3ebf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:14:41.223769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d31f36ef-30ea-4e78-a92b-8b30b6993442 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:41.224428 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d31f36ef-30ea-4e78-a92b-8b30b6993442 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:14:41.225994 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1182/2377] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:14:41 2022] DELETE /identity/v3/projects/09b0437472484bb4ac61f95e5f4c3ebf => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:14:41.231639 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625d2cb-dfe1-4c14-96b0-b253b0bf0b54 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:41.232203 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625d2cb-dfe1-4c14-96b0-b253b0bf0b54 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:41.232203 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f625d2cb-dfe1-4c14-96b0-b253b0bf0b54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:41.251265 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-f625d2cb-dfe1-4c14-96b0-b253b0bf0b54 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:14:41.252472 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f625d2cb-dfe1-4c14-96b0-b253b0bf0b54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:14:41.311317 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1196/2378] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:14:41.319926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdd0ab-9212-44a6-b553-20a83af1d157 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:41.320209 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdd0ab-9212-44a6-b553-20a83af1d157 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:41.320488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcdd0ab-9212-44a6-b553-20a83af1d157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:41.339548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-2dcdd0ab-9212-44a6-b553-20a83af1d157 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:14:41.340791 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2dcdd0ab-9212-44a6-b553-20a83af1d157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:14:41.349094 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1183/2379] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:14:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:14:41.353394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625d2cb-dfe1-4c14-96b0-b253b0bf0b54 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:14:41.363247 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f625d2cb-dfe1-4c14-96b0-b253b0bf0b54 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:14:41.367898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a80462-4bc7-4e97-b261-1149aba730e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:14:41.368555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a80462-4bc7-4e97-b261-1149aba730e9 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:41.368839 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a80462-4bc7-4e97-b261-1149aba730e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:41.369117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a80462-4bc7-4e97-b261-1149aba730e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:41.369857 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a80462-4bc7-4e97-b261-1149aba730e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:14:41.370165 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a80462-4bc7-4e97-b261-1149aba730e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:14:41.372381 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a80462-4bc7-4e97-b261-1149aba730e9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:14:41.384900 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1197/2380] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:14:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:14:41.390173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdd0ab-9212-44a6-b553-20a83af1d157 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:14:41.399763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcdd0ab-9212-44a6-b553-20a83af1d157 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:14:41.404753 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6f8bf-a258-4633-880c-5743f6c2a1d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:14:41.405616 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6f8bf-a258-4633-880c-5743f6c2a1d8 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:41.405762 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6f8bf-a258-4633-880c-5743f6c2a1d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:41.405964 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b6f8bf-a258-4633-880c-5743f6c2a1d8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:41.406724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b6f8bf-a258-4633-880c-5743f6c2a1d8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:14:41.408623 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b6f8bf-a258-4633-880c-5743f6c2a1d8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:14:41.437600 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1184/2381] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:14:41 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:14:41.441708 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a80462-4bc7-4e97-b261-1149aba730e9 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:14:41.452214 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a80462-4bc7-4e97-b261-1149aba730e9 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:14:41.479723 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:14:41.516187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb4d9b-4f8f-4a5b-bb45-8109d3433db6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:14:41.516877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb4d9b-4f8f-4a5b-bb45-8109d3433db6 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:41.517308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb4d9b-4f8f-4a5b-bb45-8109d3433db6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:41.517622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbb4d9b-4f8f-4a5b-bb45-8109d3433db6 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:41.518354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb4d9b-4f8f-4a5b-bb45-8109d3433db6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:14:41.520301 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbb4d9b-4f8f-4a5b-bb45-8109d3433db6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:14:41.541106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1198/2382] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:14:41 2022] POST /identity/v3/users => generated 288 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:14:41.545705 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6f8bf-a258-4633-880c-5743f6c2a1d8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:14:41.556485 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b6f8bf-a258-4633-880c-5743f6c2a1d8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:14:41.561675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e90a33-68a1-49c4-a378-9f3b9025dba0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:14:41.562376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e90a33-68a1-49c4-a378-9f3b9025dba0 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:41.562660 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e90a33-68a1-49c4-a378-9f3b9025dba0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:41.562933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e90a33-68a1-49c4-a378-9f3b9025dba0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:41.563666 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e90a33-68a1-49c4-a378-9f3b9025dba0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:14:41.565326 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e90a33-68a1-49c4-a378-9f3b9025dba0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:14:41.576534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1185/2383] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:14:41.581011 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb4d9b-4f8f-4a5b-bb45-8109d3433db6 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:14:41.591994 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbb4d9b-4f8f-4a5b-bb45-8109d3433db6 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:14:41.597156 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aadc70c5-8ff6-4d40-aa5c-21a9c30eeaf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:14:41.597817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc70c5-8ff6-4d40-aa5c-21a9c30eeaf3 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:41.598097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc70c5-8ff6-4d40-aa5c-21a9c30eeaf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:41.598402 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aadc70c5-8ff6-4d40-aa5c-21a9c30eeaf3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:41.599149 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc70c5-8ff6-4d40-aa5c-21a9c30eeaf3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:14:41.600746 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aadc70c5-8ff6-4d40-aa5c-21a9c30eeaf3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:14:41.612021 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1199/2384] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:14:41.616438 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e90a33-68a1-49c4-a378-9f3b9025dba0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:14:41.628552 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e90a33-68a1-49c4-a378-9f3b9025dba0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:14:41.633445 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6039b7-f720-4712-b501-5f2559533381 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:14:41.634357 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6039b7-f720-4712-b501-5f2559533381 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:41.634643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6039b7-f720-4712-b501-5f2559533381 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:41.634918 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6039b7-f720-4712-b501-5f2559533381 admin admin] PATH_INFO: `/v3/projects/fa5a4a2891464b3c8455739df2a2888b/users/517adfc4990d42559b8e1ec1aa7aa670/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:41.650670 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6039b7-f720-4712-b501-5f2559533381 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa5a4a2891464b3c8455739df2a2888b, user_id=517adfc4990d42559b8e1ec1aa7aa670, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:14:41.653221 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6039b7-f720-4712-b501-5f2559533381 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:14:41.666626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1186/2385] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:14:41 2022] PUT /identity/v3/projects/fa5a4a2891464b3c8455739df2a2888b/users/517adfc4990d42559b8e1ec1aa7aa670/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:14:41.678351 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a9b34-5539-4b5f-bc32-46bc05a207d5 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:41.678755 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a9b34-5539-4b5f-bc32-46bc05a207d5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:41.679244 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-122a9b34-5539-4b5f-bc32-46bc05a207d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:41.707102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-122a9b34-5539-4b5f-bc32-46bc05a207d5 None None] MFA Rules not processed for user `517adfc4990d42559b8e1ec1aa7aa670`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:14:41.708350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-122a9b34-5539-4b5f-bc32-46bc05a207d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:14:41.778524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1200/2386] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:14:41 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:14:41.783116 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6039b7-f720-4712-b501-5f2559533381 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:14:41.852802 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6039b7-f720-4712-b501-5f2559533381 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:14:41.858435 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f2859-2541-444c-9bdd-560890f760f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:14:41.859345 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f2859-2541-444c-9bdd-560890f760f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:41.859679 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f2859-2541-444c-9bdd-560890f760f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:41.859966 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938f2859-2541-444c-9bdd-560890f760f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:41.860842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f2859-2541-444c-9bdd-560890f760f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:14:41.862921 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938f2859-2541-444c-9bdd-560890f760f4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:14:41.890394 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1187/2387] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:14:41 2022] POST /identity/v3/projects => generated 375 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:14:41.894715 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a9b34-5539-4b5f-bc32-46bc05a207d5 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:14:41.906924 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-122a9b34-5539-4b5f-bc32-46bc05a207d5 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:14:41.964066 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb32e50-6768-472a-98b2-faf5b726f396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:14:41.965321 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb32e50-6768-472a-98b2-faf5b726f396 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:41.965608 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb32e50-6768-472a-98b2-faf5b726f396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:41.965892 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb32e50-6768-472a-98b2-faf5b726f396 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:41.966653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb32e50-6768-472a-98b2-faf5b726f396 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:14:41.968625 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb32e50-6768-472a-98b2-faf5b726f396 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:14:41.991580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1201/2388] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:14:41 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:14:41.996077 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f2859-2541-444c-9bdd-560890f760f4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:14:42.007279 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938f2859-2541-444c-9bdd-560890f760f4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:14:42.012272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac75e3-a407-4120-8f3e-29eb2815ca9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:14:42.012952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac75e3-a407-4120-8f3e-29eb2815ca9a admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:42.013247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac75e3-a407-4120-8f3e-29eb2815ca9a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:42.013526 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14ac75e3-a407-4120-8f3e-29eb2815ca9a admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:42.014345 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac75e3-a407-4120-8f3e-29eb2815ca9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:14:42.016197 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14ac75e3-a407-4120-8f3e-29eb2815ca9a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:14:42.028371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1188/2389] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:14:41 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:14:42.032743 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb32e50-6768-472a-98b2-faf5b726f396 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:14:42.043602 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb32e50-6768-472a-98b2-faf5b726f396 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:14:42.049043 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7224e5-da82-4cd9-ab21-6b43409426af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:14:42.049960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7224e5-da82-4cd9-ab21-6b43409426af admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:42.050528 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7224e5-da82-4cd9-ab21-6b43409426af admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:42.050838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7224e5-da82-4cd9-ab21-6b43409426af admin admin] PATH_INFO: `/v3/projects/f4b3bc2c44074fc8ad6aadbd549e1f59/users/f1cc4d58fae24d14871ce4ab43277a21/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:42.066918 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7224e5-da82-4cd9-ab21-6b43409426af admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b3bc2c44074fc8ad6aadbd549e1f59, user_id=f1cc4d58fae24d14871ce4ab43277a21, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:14:42.069606 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7224e5-da82-4cd9-ab21-6b43409426af admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:14:42.083811 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1202/2390] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:14:42 2022] PUT /identity/v3/projects/f4b3bc2c44074fc8ad6aadbd549e1f59/users/f1cc4d58fae24d14871ce4ab43277a21/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:14:42.088511 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac75e3-a407-4120-8f3e-29eb2815ca9a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:14:42.099334 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14ac75e3-a407-4120-8f3e-29eb2815ca9a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:14:42.120913 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:14:42.162241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04f26ca9-da7a-48bf-861c-f22200a6c6b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:14:42.162867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f26ca9-da7a-48bf-861c-f22200a6c6b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:42.163158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f26ca9-da7a-48bf-861c-f22200a6c6b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:42.163440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04f26ca9-da7a-48bf-861c-f22200a6c6b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:42.164192 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f26ca9-da7a-48bf-861c-f22200a6c6b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:14:42.165830 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04f26ca9-da7a-48bf-861c-f22200a6c6b1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:14:42.176506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1189/2391] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:14:42.180996 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7224e5-da82-4cd9-ab21-6b43409426af admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:14:42.186562 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:14:42.193074 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7224e5-da82-4cd9-ab21-6b43409426af admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:14:42.199009 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10fb6f5-d98f-4a1c-aa1f-d37eb065ecab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:14:42.200399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10fb6f5-d98f-4a1c-aa1f-d37eb065ecab admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:42.200708 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10fb6f5-d98f-4a1c-aa1f-d37eb065ecab admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:42.200985 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d10fb6f5-d98f-4a1c-aa1f-d37eb065ecab admin admin] PATH_INFO: `/v3/projects/f4b3bc2c44074fc8ad6aadbd549e1f59/users/f1cc4d58fae24d14871ce4ab43277a21/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:42.203411 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10fb6f5-d98f-4a1c-aa1f-d37eb065ecab admin admin] RBAC: Authorizing `identity:create_grant(project_id=f4b3bc2c44074fc8ad6aadbd549e1f59, user_id=f1cc4d58fae24d14871ce4ab43277a21, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:14:42.206514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d10fb6f5-d98f-4a1c-aa1f-d37eb065ecab admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:14:42.218083 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1203/2392] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:14:42 2022] PUT /identity/v3/projects/f4b3bc2c44074fc8ad6aadbd549e1f59/users/f1cc4d58fae24d14871ce4ab43277a21/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:14:42.228385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457c01d-08fa-4dae-b1c0-161f9d12c3bc None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:42.228706 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457c01d-08fa-4dae-b1c0-161f9d12c3bc None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:42.228992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5457c01d-08fa-4dae-b1c0-161f9d12c3bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:42.257119 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5457c01d-08fa-4dae-b1c0-161f9d12c3bc None None] MFA Rules not processed for user `f1cc4d58fae24d14871ce4ab43277a21`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:14:42.258374 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5457c01d-08fa-4dae-b1c0-161f9d12c3bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:14:42.331601 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1190/2393] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:14:42 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:14:42.343467 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10fb6f5-d98f-4a1c-aa1f-d37eb065ecab admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:14:42.345609 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d10fb6f5-d98f-4a1c-aa1f-d37eb065ecab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:14:42.411061 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d10fb6f5-d98f-4a1c-aa1f-d37eb065ecab admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:14:42.415214 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f82f8-57a5-4d36-86f8-01814aab8547 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:14:42.415808 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f82f8-57a5-4d36-86f8-01814aab8547 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:42.416092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f82f8-57a5-4d36-86f8-01814aab8547 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:42.416378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-666f82f8-57a5-4d36-86f8-01814aab8547 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:42.484584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f82f8-57a5-4d36-86f8-01814aab8547 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:14:42.486198 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-666f82f8-57a5-4d36-86f8-01814aab8547 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:14:42.497560 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1204/2394] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:14:42 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:14:49.331054 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457c01d-08fa-4dae-b1c0-161f9d12c3bc None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:14:49.352059 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5457c01d-08fa-4dae-b1c0-161f9d12c3bc None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:14:49.356426 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3836f15-adf5-4667-9953-44b4c063600c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:14:49.357082 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3836f15-adf5-4667-9953-44b4c063600c service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:14:49.357368 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3836f15-adf5-4667-9953-44b4c063600c service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:14:49.357646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3836f15-adf5-4667-9953-44b4c063600c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:14:49.387616 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3836f15-adf5-4667-9953-44b4c063600c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:14:49.389071 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3836f15-adf5-4667-9953-44b4c063600c service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:14:49.398993 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1191/2395] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:14:49 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 69 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:14:52.491884 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=52,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:14:59.394533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=72 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:02.023600 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f82f8-57a5-4d36-86f8-01814aab8547 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:02.025006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-666f82f8-57a5-4d36-86f8-01814aab8547 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:02.174326 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-666f82f8-57a5-4d36-86f8-01814aab8547 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:02.179823 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f08cd-ddf6-4b28-9189-883da10ecfb0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee29b7f1cea64d6781e1df5c8021363f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:02.180489 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f08cd-ddf6-4b28-9189-883da10ecfb0 service placement] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:02.180774 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f08cd-ddf6-4b28-9189-883da10ecfb0 service placement] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:02.181056 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e93f08cd-ddf6-4b28-9189-883da10ecfb0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:02.182923 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e93f08cd-ddf6-4b28-9189-883da10ecfb0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:02.195370 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f08cd-ddf6-4b28-9189-883da10ecfb0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:02.196973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e93f08cd-ddf6-4b28-9189-883da10ecfb0 service placement] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:02.207163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1205/2396] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:15:02 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 184 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:15:06.745950 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3836f15-adf5-4667-9953-44b4c063600c service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:06.747394 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c3836f15-adf5-4667-9953-44b4c063600c service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:06.827215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3836f15-adf5-4667-9953-44b4c063600c service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:06.830952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82fc025e-b804-4354-9dd4-b49da283947d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:06.831641 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc025e-b804-4354-9dd4-b49da283947d service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:06.831929 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc025e-b804-4354-9dd4-b49da283947d service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:06.832211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82fc025e-b804-4354-9dd4-b49da283947d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:06.833994 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-82fc025e-b804-4354-9dd4-b49da283947d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:06.894097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc025e-b804-4354-9dd4-b49da283947d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:06.895561 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82fc025e-b804-4354-9dd4-b49da283947d service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:06.906364 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1192/2397] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:15:06 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 161 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:15:12.202738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=60 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:16.900454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=57 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:18.880937 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f08cd-ddf6-4b28-9189-883da10ecfb0 service placement] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:18.891868 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e93f08cd-ddf6-4b28-9189-883da10ecfb0 service placement] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:18.895652 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f22b9-5590-4ddb-a2cb-aa4a841ba5ca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:18.896389 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f22b9-5590-4ddb-a2cb-aa4a841ba5ca service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:18.896922 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f22b9-5590-4ddb-a2cb-aa4a841ba5ca service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:18.897297 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180f22b9-5590-4ddb-a2cb-aa4a841ba5ca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:18.899218 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-180f22b9-5590-4ddb-a2cb-aa4a841ba5ca service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:19.005132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f22b9-5590-4ddb-a2cb-aa4a841ba5ca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:19.006724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180f22b9-5590-4ddb-a2cb-aa4a841ba5ca service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:19.028210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1206/2398] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:15:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:15:20.724865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b1a23-ea3d-465b-ba55-3faaacc6717b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:20.725501 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b1a23-ea3d-465b-ba55-3faaacc6717b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:20.726628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-082b1a23-ea3d-465b-ba55-3faaacc6717b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:20.746036 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-082b1a23-ea3d-465b-ba55-3faaacc6717b None None] MFA Rules not processed for user `04e86369906d4633b02a5ca1bcc24b35`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:20.747646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-082b1a23-ea3d-465b-ba55-3faaacc6717b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:20.816342 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1193/2399] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:20 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:15:21.615765 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f22b9-5590-4ddb-a2cb-aa4a841ba5ca service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:21.626752 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180f22b9-5590-4ddb-a2cb-aa4a841ba5ca service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:21.632576 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910cd20-d755-402c-866b-d0cdc32e6a22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:21.633726 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910cd20-d755-402c-866b-d0cdc32e6a22 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:21.634184 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910cd20-d755-402c-866b-d0cdc32e6a22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:21.634670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d910cd20-d755-402c-866b-d0cdc32e6a22 admin admin] PATH_INFO: `/v3/users/517adfc4990d42559b8e1ec1aa7aa670` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:21.636688 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910cd20-d755-402c-866b-d0cdc32e6a22 admin admin] RBAC: Authorizing `identity:delete_user(user_id=517adfc4990d42559b8e1ec1aa7aa670)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:21.639079 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d910cd20-d755-402c-866b-d0cdc32e6a22 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:21.706104 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d910cd20-d755-402c-866b-d0cdc32e6a22 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '517adfc4990d42559b8e1ec1aa7aa670'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:21.722995 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d910cd20-d755-402c-866b-d0cdc32e6a22 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '517adfc4990d42559b8e1ec1aa7aa670'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:21.736893 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d910cd20-d755-402c-866b-d0cdc32e6a22 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '517adfc4990d42559b8e1ec1aa7aa670'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:21.737803 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d910cd20-d755-402c-866b-d0cdc32e6a22 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '517adfc4990d42559b8e1ec1aa7aa670'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:21.771687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d910cd20-d755-402c-866b-d0cdc32e6a22 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '517adfc4990d42559b8e1ec1aa7aa670'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:21.779942 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1207/2400] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:15:21 2022] DELETE /identity/v3/users/517adfc4990d42559b8e1ec1aa7aa670 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:21.784365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e563c81-1bdf-498c-a781-3cd63f89444f None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:21.785406 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e563c81-1bdf-498c-a781-3cd63f89444f None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:21.785883 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e563c81-1bdf-498c-a781-3cd63f89444f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:21.823450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0e563c81-1bdf-498c-a781-3cd63f89444f None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:21.827510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0e563c81-1bdf-498c-a781-3cd63f89444f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:21.921297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1194/2401] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:15:21.933077 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910cd20-d755-402c-866b-d0cdc32e6a22 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:21.934595 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d910cd20-d755-402c-866b-d0cdc32e6a22 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:22.003322 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:22.023135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d910cd20-d755-402c-866b-d0cdc32e6a22 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:22.028511 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab133c-ad09-4bb4-b7c8-0944c3891ce1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:22.029220 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab133c-ad09-4bb4-b7c8-0944c3891ce1 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:22.029513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab133c-ad09-4bb4-b7c8-0944c3891ce1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:22.029787 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ab133c-ad09-4bb4-b7c8-0944c3891ce1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:22.040081 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab133c-ad09-4bb4-b7c8-0944c3891ce1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:22.043182 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ab133c-ad09-4bb4-b7c8-0944c3891ce1 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:22.057973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1208/2402] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:15:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:15:22.376907 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e563c81-1bdf-498c-a781-3cd63f89444f None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:22.378499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0e563c81-1bdf-498c-a781-3cd63f89444f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:22.395170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e563c81-1bdf-498c-a781-3cd63f89444f None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:22.401711 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52539885-2ca2-4154-9aa1-7517c62cca68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:22.402777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52539885-2ca2-4154-9aa1-7517c62cca68 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:22.403315 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52539885-2ca2-4154-9aa1-7517c62cca68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:22.403315 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52539885-2ca2-4154-9aa1-7517c62cca68 admin admin] PATH_INFO: `/v3/projects/fa5a4a2891464b3c8455739df2a2888b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:22.404730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52539885-2ca2-4154-9aa1-7517c62cca68 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa5a4a2891464b3c8455739df2a2888b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:22.406774 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52539885-2ca2-4154-9aa1-7517c62cca68 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:22.477715 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-52539885-2ca2-4154-9aa1-7517c62cca68 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa5a4a2891464b3c8455739df2a2888b', 'request_id': 'req-52539885-2ca2-4154-9aa1-7517c62cca68'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:22.485714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-52539885-2ca2-4154-9aa1-7517c62cca68 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa5a4a2891464b3c8455739df2a2888b', 'request_id': 'req-52539885-2ca2-4154-9aa1-7517c62cca68'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:22.495497 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-52539885-2ca2-4154-9aa1-7517c62cca68 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa5a4a2891464b3c8455739df2a2888b', 'request_id': 'req-52539885-2ca2-4154-9aa1-7517c62cca68'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:22.511358 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-52539885-2ca2-4154-9aa1-7517c62cca68 admin admin] The token cache is being invalidate because project fa5a4a2891464b3c8455739df2a2888b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:15:22.511800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-52539885-2ca2-4154-9aa1-7517c62cca68 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:22.512527 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-52539885-2ca2-4154-9aa1-7517c62cca68 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:22.514238 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1195/2403] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:15:22 2022] DELETE /identity/v3/projects/fa5a4a2891464b3c8455739df2a2888b => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:22.518197 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab133c-ad09-4bb4-b7c8-0944c3891ce1 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:22.529732 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ab133c-ad09-4bb4-b7c8-0944c3891ce1 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:22.649586 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61cefe48-aaad-4ca1-a979-65854303de72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:22.650593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cefe48-aaad-4ca1-a979-65854303de72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:22.651450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cefe48-aaad-4ca1-a979-65854303de72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:22.651738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61cefe48-aaad-4ca1-a979-65854303de72 admin admin] PATH_INFO: `/v3/users/f1cc4d58fae24d14871ce4ab43277a21` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:22.688866 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cefe48-aaad-4ca1-a979-65854303de72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f1cc4d58fae24d14871ce4ab43277a21)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:22.691095 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61cefe48-aaad-4ca1-a979-65854303de72 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:22.795943 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-61cefe48-aaad-4ca1-a979-65854303de72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f1cc4d58fae24d14871ce4ab43277a21'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:22.822115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-61cefe48-aaad-4ca1-a979-65854303de72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f1cc4d58fae24d14871ce4ab43277a21'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:22.833979 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-61cefe48-aaad-4ca1-a979-65854303de72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f1cc4d58fae24d14871ce4ab43277a21'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:22.835247 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-61cefe48-aaad-4ca1-a979-65854303de72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f1cc4d58fae24d14871ce4ab43277a21'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:22.846878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-61cefe48-aaad-4ca1-a979-65854303de72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f1cc4d58fae24d14871ce4ab43277a21'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:22.866206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1209/2404] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:15:22 2022] DELETE /identity/v3/users/f1cc4d58fae24d14871ce4ab43277a21 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:23.219992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52539885-2ca2-4154-9aa1-7517c62cca68 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:23.222228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-52539885-2ca2-4154-9aa1-7517c62cca68 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:23.240230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52539885-2ca2-4154-9aa1-7517c62cca68 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:23.265621 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:23.322849 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5153e8-b0c6-496b-a5b1-9a53ddb9abe1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:23.323811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5153e8-b0c6-496b-a5b1-9a53ddb9abe1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:23.324094 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5153e8-b0c6-496b-a5b1-9a53ddb9abe1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:23.324536 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5153e8-b0c6-496b-a5b1-9a53ddb9abe1 admin admin] PATH_INFO: `/v3/projects/f4b3bc2c44074fc8ad6aadbd549e1f59` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:23.325955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5153e8-b0c6-496b-a5b1-9a53ddb9abe1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f4b3bc2c44074fc8ad6aadbd549e1f59)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:23.333600 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5153e8-b0c6-496b-a5b1-9a53ddb9abe1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:23.384224 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0a5153e8-b0c6-496b-a5b1-9a53ddb9abe1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f4b3bc2c44074fc8ad6aadbd549e1f59', 'request_id': 'req-0a5153e8-b0c6-496b-a5b1-9a53ddb9abe1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:23.391654 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0a5153e8-b0c6-496b-a5b1-9a53ddb9abe1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f4b3bc2c44074fc8ad6aadbd549e1f59', 'request_id': 'req-0a5153e8-b0c6-496b-a5b1-9a53ddb9abe1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:23.401830 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0a5153e8-b0c6-496b-a5b1-9a53ddb9abe1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f4b3bc2c44074fc8ad6aadbd549e1f59', 'request_id': 'req-0a5153e8-b0c6-496b-a5b1-9a53ddb9abe1'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:23.414184 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0a5153e8-b0c6-496b-a5b1-9a53ddb9abe1 admin admin] The token cache is being invalidate because project f4b3bc2c44074fc8ad6aadbd549e1f59 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:15:23.414666 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0a5153e8-b0c6-496b-a5b1-9a53ddb9abe1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:23.415373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0a5153e8-b0c6-496b-a5b1-9a53ddb9abe1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:23.417493 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1196/2405] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:15:23 2022] DELETE /identity/v3/projects/f4b3bc2c44074fc8ad6aadbd549e1f59 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:23.422344 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffc001-33f9-444a-9132-1421582279d0 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:23.422591 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffc001-33f9-444a-9132-1421582279d0 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:23.422874 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3ffc001-33f9-444a-9132-1421582279d0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:23.480270 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-f3ffc001-33f9-444a-9132-1421582279d0 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:23.481584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f3ffc001-33f9-444a-9132-1421582279d0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:23.536103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:23.576399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1210/2406] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:15:23.584721 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0ac73-2a11-4dc9-94da-c0ce120ca9de None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:23.585264 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0ac73-2a11-4dc9-94da-c0ce120ca9de None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:23.585726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d0ac73-2a11-4dc9-94da-c0ce120ca9de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:23.607724 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c8d0ac73-2a11-4dc9-94da-c0ce120ca9de None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:23.609263 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c8d0ac73-2a11-4dc9-94da-c0ce120ca9de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:23.617737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1197/2407] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:15:23.622399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffc001-33f9-444a-9132-1421582279d0 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:23.634012 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3ffc001-33f9-444a-9132-1421582279d0 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:23.640408 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d930e8-33e0-4d2f-95c8-d2c78818ecc8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:23.641099 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d930e8-33e0-4d2f-95c8-d2c78818ecc8 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:23.641393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d930e8-33e0-4d2f-95c8-d2c78818ecc8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:23.641680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33d930e8-33e0-4d2f-95c8-d2c78818ecc8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:23.642700 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d930e8-33e0-4d2f-95c8-d2c78818ecc8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:15:23.643022 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d930e8-33e0-4d2f-95c8-d2c78818ecc8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:23.644646 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33d930e8-33e0-4d2f-95c8-d2c78818ecc8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:23.661522 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1211/2408] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:15:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:15:23.663492 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0ac73-2a11-4dc9-94da-c0ce120ca9de None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:23.674459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d0ac73-2a11-4dc9-94da-c0ce120ca9de None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:23.680845 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ddfb7-f461-4000-a97d-6da2ff4ae30a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:23.681670 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ddfb7-f461-4000-a97d-6da2ff4ae30a admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:23.681843 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ddfb7-f461-4000-a97d-6da2ff4ae30a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:23.682141 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce5ddfb7-f461-4000-a97d-6da2ff4ae30a admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:23.683472 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ddfb7-f461-4000-a97d-6da2ff4ae30a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:23.685252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce5ddfb7-f461-4000-a97d-6da2ff4ae30a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:23.719120 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1198/2409] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:15:23 2022] POST /identity/v3/projects => generated 407 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:15:23.723924 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d930e8-33e0-4d2f-95c8-d2c78818ecc8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:23.739873 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33d930e8-33e0-4d2f-95c8-d2c78818ecc8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:23.814341 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3279b-b3fc-4c3b-89fd-3f9d1d358ba7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:23.814341 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3279b-b3fc-4c3b-89fd-3f9d1d358ba7 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:23.814341 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3279b-b3fc-4c3b-89fd-3f9d1d358ba7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:23.814341 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b3279b-b3fc-4c3b-89fd-3f9d1d358ba7 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:23.814745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3279b-b3fc-4c3b-89fd-3f9d1d358ba7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:23.816286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b3279b-b3fc-4c3b-89fd-3f9d1d358ba7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:23.839197 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1212/2410] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:15:23 2022] POST /identity/v3/users => generated 304 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:15:23.843866 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ddfb7-f461-4000-a97d-6da2ff4ae30a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:23.855969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce5ddfb7-f461-4000-a97d-6da2ff4ae30a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:23.862838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997faa06-874c-48b0-9246-f758e37dc88e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:23.863526 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997faa06-874c-48b0-9246-f758e37dc88e admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:23.863857 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997faa06-874c-48b0-9246-f758e37dc88e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:23.864165 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997faa06-874c-48b0-9246-f758e37dc88e admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:23.864902 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997faa06-874c-48b0-9246-f758e37dc88e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:23.866606 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997faa06-874c-48b0-9246-f758e37dc88e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:23.882375 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1199/2411] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:15:23.886372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3279b-b3fc-4c3b-89fd-3f9d1d358ba7 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:23.900905 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b3279b-b3fc-4c3b-89fd-3f9d1d358ba7 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:23.909404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51784d80-c372-4e9f-8449-a07beb2abab3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:23.910150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51784d80-c372-4e9f-8449-a07beb2abab3 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:23.910473 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51784d80-c372-4e9f-8449-a07beb2abab3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:23.910756 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51784d80-c372-4e9f-8449-a07beb2abab3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:23.911508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51784d80-c372-4e9f-8449-a07beb2abab3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:23.913126 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51784d80-c372-4e9f-8449-a07beb2abab3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:23.931400 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1213/2412] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:15:23 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:15:23.931805 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997faa06-874c-48b0-9246-f758e37dc88e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:23.944492 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997faa06-874c-48b0-9246-f758e37dc88e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:23.951154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48cbe6d-3e0e-4800-92a6-c70c346c8626 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:23.951714 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cbe6d-3e0e-4800-92a6-c70c346c8626 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:23.951994 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cbe6d-3e0e-4800-92a6-c70c346c8626 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:23.952277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e48cbe6d-3e0e-4800-92a6-c70c346c8626 admin admin] PATH_INFO: `/v3/projects/bfae43c4a76d45a29563f347fd63e7e5/users/d592d865f96944caa9cf1508af654332/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:23.978979 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48cbe6d-3e0e-4800-92a6-c70c346c8626 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bfae43c4a76d45a29563f347fd63e7e5, user_id=d592d865f96944caa9cf1508af654332, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:23.981796 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e48cbe6d-3e0e-4800-92a6-c70c346c8626 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:24.007416 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1200/2413] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:15:23 2022] PUT /identity/v3/projects/bfae43c4a76d45a29563f347fd63e7e5/users/d592d865f96944caa9cf1508af654332/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:24.009351 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85498e-c458-47a2-a274-4a83d912a8fa None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:24.009751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85498e-c458-47a2-a274-4a83d912a8fa None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:24.010193 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d85498e-c458-47a2-a274-4a83d912a8fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:24.041838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4d85498e-c458-47a2-a274-4a83d912a8fa None None] MFA Rules not processed for user `d592d865f96944caa9cf1508af654332`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:24.043519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4d85498e-c458-47a2-a274-4a83d912a8fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:24.143619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1214/2414] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:24 2022] POST /identity/v3/auth/tokens => generated 3274 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:15:24.147413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48cbe6d-3e0e-4800-92a6-c70c346c8626 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:24.204475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:24.224979 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e48cbe6d-3e0e-4800-92a6-c70c346c8626 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:24.235082 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579656c1-84cf-4a2e-9d56-934ff64a3f73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:24.235862 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579656c1-84cf-4a2e-9d56-934ff64a3f73 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:24.237521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579656c1-84cf-4a2e-9d56-934ff64a3f73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:24.237830 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579656c1-84cf-4a2e-9d56-934ff64a3f73 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:24.238591 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579656c1-84cf-4a2e-9d56-934ff64a3f73 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:24.240530 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-579656c1-84cf-4a2e-9d56-934ff64a3f73 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:24.275324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1201/2415] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:15:24 2022] POST /identity/v3/projects => generated 409 bytes in 125 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:15:24.276878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85498e-c458-47a2-a274-4a83d912a8fa None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:24.288372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d85498e-c458-47a2-a274-4a83d912a8fa None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:24.324740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:24.365765 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19ef38-983c-483c-9d50-ae818535d938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:24.367065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19ef38-983c-483c-9d50-ae818535d938 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:24.367355 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19ef38-983c-483c-9d50-ae818535d938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:24.367741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e19ef38-983c-483c-9d50-ae818535d938 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:24.369021 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e19ef38-983c-483c-9d50-ae818535d938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:24.372130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e19ef38-983c-483c-9d50-ae818535d938 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:24.399023 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1215/2416] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:15:24 2022] POST /identity/v3/users => generated 311 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:15:24.403090 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579656c1-84cf-4a2e-9d56-934ff64a3f73 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:24.414346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579656c1-84cf-4a2e-9d56-934ff64a3f73 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:24.420039 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3458663-2a72-4cd0-98ea-ebc156feed2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:24.421371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3458663-2a72-4cd0-98ea-ebc156feed2b admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:24.421371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3458663-2a72-4cd0-98ea-ebc156feed2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:24.421371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3458663-2a72-4cd0-98ea-ebc156feed2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:24.421999 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3458663-2a72-4cd0-98ea-ebc156feed2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:24.423641 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3458663-2a72-4cd0-98ea-ebc156feed2b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:24.436788 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1202/2417] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:15:24.441051 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19ef38-983c-483c-9d50-ae818535d938 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:24.451581 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e19ef38-983c-483c-9d50-ae818535d938 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:24.456591 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09561a1f-5aac-44d5-913a-14ab0d85b1cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:24.457595 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09561a1f-5aac-44d5-913a-14ab0d85b1cb admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:24.458371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09561a1f-5aac-44d5-913a-14ab0d85b1cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:24.458371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09561a1f-5aac-44d5-913a-14ab0d85b1cb admin admin] PATH_INFO: `/v3/projects/4cdc2f0f7347413d900b484a98322970/users/e8c3cf541f004c429a6d96812dc6a008/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:24.480506 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09561a1f-5aac-44d5-913a-14ab0d85b1cb admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cdc2f0f7347413d900b484a98322970, user_id=e8c3cf541f004c429a6d96812dc6a008, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:24.496621 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09561a1f-5aac-44d5-913a-14ab0d85b1cb admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:24.539594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1216/2418] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:15:24 2022] PUT /identity/v3/projects/4cdc2f0f7347413d900b484a98322970/users/e8c3cf541f004c429a6d96812dc6a008/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 99 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:24.545338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3458663-2a72-4cd0-98ea-ebc156feed2b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:24.562932 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3458663-2a72-4cd0-98ea-ebc156feed2b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:24.671112 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-897ed604-5a65-4aa7-b0ec-8c59710cda0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:24.671963 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ed604-5a65-4aa7-b0ec-8c59710cda0d admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:24.672357 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ed604-5a65-4aa7-b0ec-8c59710cda0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:24.672769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-897ed604-5a65-4aa7-b0ec-8c59710cda0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:24.673646 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ed604-5a65-4aa7-b0ec-8c59710cda0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:24.676249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-897ed604-5a65-4aa7-b0ec-8c59710cda0d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:24.697408 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1203/2419] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:15:24 2022] GET /identity/v3/roles => generated 1420 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:15:24.701314 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09561a1f-5aac-44d5-913a-14ab0d85b1cb admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:24.722638 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09561a1f-5aac-44d5-913a-14ab0d85b1cb admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:24.729471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5550f3-b5c8-4863-92de-931ed36d9034 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:24.730425 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5550f3-b5c8-4863-92de-931ed36d9034 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:24.732004 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5550f3-b5c8-4863-92de-931ed36d9034 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:24.732931 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5550f3-b5c8-4863-92de-931ed36d9034 admin admin] PATH_INFO: `/v3/projects/4cdc2f0f7347413d900b484a98322970/users/e8c3cf541f004c429a6d96812dc6a008/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:24.737150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5550f3-b5c8-4863-92de-931ed36d9034 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cdc2f0f7347413d900b484a98322970, user_id=e8c3cf541f004c429a6d96812dc6a008, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:24.740509 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5550f3-b5c8-4863-92de-931ed36d9034 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:24.755696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1217/2420] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:15:24 2022] PUT /identity/v3/projects/4cdc2f0f7347413d900b484a98322970/users/e8c3cf541f004c429a6d96812dc6a008/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:24.837258 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ff964-89f3-4289-a991-9cd590d28416 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:24.837569 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ff964-89f3-4289-a991-9cd590d28416 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:24.840237 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295ff964-89f3-4289-a991-9cd590d28416 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:24.895717 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-295ff964-89f3-4289-a991-9cd590d28416 None None] MFA Rules not processed for user `e8c3cf541f004c429a6d96812dc6a008`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:24.898236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-295ff964-89f3-4289-a991-9cd590d28416 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:25.025450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1204/2421] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:24 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:15:25.037851 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5550f3-b5c8-4863-92de-931ed36d9034 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:25.039979 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7c5550f3-b5c8-4863-92de-931ed36d9034 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:25.154636 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5550f3-b5c8-4863-92de-931ed36d9034 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:25.158805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4911ac-b31d-45f0-8d1b-7d21e8984bbd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:25.159588 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4911ac-b31d-45f0-8d1b-7d21e8984bbd service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:25.159949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4911ac-b31d-45f0-8d1b-7d21e8984bbd service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:25.160305 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e4911ac-b31d-45f0-8d1b-7d21e8984bbd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:25.174289 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4911ac-b31d-45f0-8d1b-7d21e8984bbd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:25.176237 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e4911ac-b31d-45f0-8d1b-7d21e8984bbd service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:25.194463 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1218/2422] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:15:25 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:15:25.400514 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ff964-89f3-4289-a991-9cd590d28416 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:25.413133 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295ff964-89f3-4289-a991-9cd590d28416 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:25.418154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5814e-38cc-4780-8977-749a65d6ec9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:25.419602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5814e-38cc-4780-8977-749a65d6ec9d service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:25.419602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5814e-38cc-4780-8977-749a65d6ec9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:25.419808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da5814e-38cc-4780-8977-749a65d6ec9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:25.471535 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:25.499989 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da5814e-38cc-4780-8977-749a65d6ec9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:25.499989 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da5814e-38cc-4780-8977-749a65d6ec9d service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:25.505124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1205/2423] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:15:25 2022] GET /identity/v3/auth/tokens => generated 3274 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:15:27.398651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4911ac-b31d-45f0-8d1b-7d21e8984bbd service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:27.400069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9e4911ac-b31d-45f0-8d1b-7d21e8984bbd service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:27.467039 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e4911ac-b31d-45f0-8d1b-7d21e8984bbd service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:27.471470 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd26655-eb1f-42ca-abe9-6fd6e0c49454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:27.472325 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd26655-eb1f-42ca-abe9-6fd6e0c49454 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:27.472602 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd26655-eb1f-42ca-abe9-6fd6e0c49454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:27.472874 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd26655-eb1f-42ca-abe9-6fd6e0c49454 admin admin] PATH_INFO: `/v3/users/e12ac3751780495081b2ef41d13ba360` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:27.474044 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd26655-eb1f-42ca-abe9-6fd6e0c49454 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e12ac3751780495081b2ef41d13ba360)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:27.475982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd26655-eb1f-42ca-abe9-6fd6e0c49454 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:27.482997 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:27.523607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dbd26655-eb1f-42ca-abe9-6fd6e0c49454 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e12ac3751780495081b2ef41d13ba360'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:27.536423 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dbd26655-eb1f-42ca-abe9-6fd6e0c49454 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e12ac3751780495081b2ef41d13ba360'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:27.544230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dbd26655-eb1f-42ca-abe9-6fd6e0c49454 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e12ac3751780495081b2ef41d13ba360'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:27.544855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dbd26655-eb1f-42ca-abe9-6fd6e0c49454 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e12ac3751780495081b2ef41d13ba360'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:27.552295 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dbd26655-eb1f-42ca-abe9-6fd6e0c49454 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e12ac3751780495081b2ef41d13ba360'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:27.559348 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1219/2424] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:15:27 2022] DELETE /identity/v3/users/e12ac3751780495081b2ef41d13ba360 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:27.730715 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5814e-38cc-4780-8977-749a65d6ec9d service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:27.743178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da5814e-38cc-4780-8977-749a65d6ec9d service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:27.805273 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8a662-de0c-454e-8aba-2140cf3b2d4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:27.806159 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8a662-de0c-454e-8aba-2140cf3b2d4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:27.806476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8a662-de0c-454e-8aba-2140cf3b2d4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:27.806757 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a8a662-de0c-454e-8aba-2140cf3b2d4b admin admin] PATH_INFO: `/v3/projects/2c01634e004b43239e6912ee0a9d1ffd` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:27.808020 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8a662-de0c-454e-8aba-2140cf3b2d4b admin admin] RBAC: Authorizing `identity:delete_project(project_id=2c01634e004b43239e6912ee0a9d1ffd)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:27.809952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a8a662-de0c-454e-8aba-2140cf3b2d4b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:27.860032 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-74a8a662-de0c-454e-8aba-2140cf3b2d4b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2c01634e004b43239e6912ee0a9d1ffd', 'request_id': 'req-74a8a662-de0c-454e-8aba-2140cf3b2d4b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:27.866359 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-74a8a662-de0c-454e-8aba-2140cf3b2d4b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2c01634e004b43239e6912ee0a9d1ffd', 'request_id': 'req-74a8a662-de0c-454e-8aba-2140cf3b2d4b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:27.875153 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-74a8a662-de0c-454e-8aba-2140cf3b2d4b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2c01634e004b43239e6912ee0a9d1ffd', 'request_id': 'req-74a8a662-de0c-454e-8aba-2140cf3b2d4b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:27.887626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-74a8a662-de0c-454e-8aba-2140cf3b2d4b admin admin] The token cache is being invalidate because project 2c01634e004b43239e6912ee0a9d1ffd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:15:27.888089 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-74a8a662-de0c-454e-8aba-2140cf3b2d4b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:27.888753 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-74a8a662-de0c-454e-8aba-2140cf3b2d4b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:27.890347 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1206/2425] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:15:27 2022] DELETE /identity/v3/projects/2c01634e004b43239e6912ee0a9d1ffd => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:15:27.894458 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd26655-eb1f-42ca-abe9-6fd6e0c49454 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:27.895977 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dbd26655-eb1f-42ca-abe9-6fd6e0c49454 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:27.910449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd26655-eb1f-42ca-abe9-6fd6e0c49454 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:27.971270 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae566dc-c345-4c55-a7f1-897b3d748176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:27.972143 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae566dc-c345-4c55-a7f1-897b3d748176 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:27.972429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae566dc-c345-4c55-a7f1-897b3d748176 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:27.972747 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae566dc-c345-4c55-a7f1-897b3d748176 admin admin] PATH_INFO: `/v3/users/24fc8e511f2d42fbbf66973fea1404f4` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:27.973905 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae566dc-c345-4c55-a7f1-897b3d748176 admin admin] RBAC: Authorizing `identity:delete_user(user_id=24fc8e511f2d42fbbf66973fea1404f4)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:27.975853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cae566dc-c345-4c55-a7f1-897b3d748176 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:28.020817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cae566dc-c345-4c55-a7f1-897b3d748176 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24fc8e511f2d42fbbf66973fea1404f4'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:28.033451 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cae566dc-c345-4c55-a7f1-897b3d748176 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24fc8e511f2d42fbbf66973fea1404f4'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:28.040841 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cae566dc-c345-4c55-a7f1-897b3d748176 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24fc8e511f2d42fbbf66973fea1404f4'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:28.041548 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cae566dc-c345-4c55-a7f1-897b3d748176 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24fc8e511f2d42fbbf66973fea1404f4'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:28.048831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cae566dc-c345-4c55-a7f1-897b3d748176 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24fc8e511f2d42fbbf66973fea1404f4'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:28.055368 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1220/2426] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:15:27 2022] DELETE /identity/v3/users/24fc8e511f2d42fbbf66973fea1404f4 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:28.230904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8a662-de0c-454e-8aba-2140cf3b2d4b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:28.232381 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-74a8a662-de0c-454e-8aba-2140cf3b2d4b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:28.247320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a8a662-de0c-454e-8aba-2140cf3b2d4b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:28.304801 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99435e2e-b910-4617-a90e-25bb418e1144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:28.305665 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99435e2e-b910-4617-a90e-25bb418e1144 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:28.305948 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99435e2e-b910-4617-a90e-25bb418e1144 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:28.306223 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99435e2e-b910-4617-a90e-25bb418e1144 admin admin] PATH_INFO: `/v3/projects/b08b03e04f394198bdf9a320a3a4064b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:28.307913 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99435e2e-b910-4617-a90e-25bb418e1144 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b08b03e04f394198bdf9a320a3a4064b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:28.309814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99435e2e-b910-4617-a90e-25bb418e1144 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:28.325659 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:28.355971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-99435e2e-b910-4617-a90e-25bb418e1144 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b08b03e04f394198bdf9a320a3a4064b', 'request_id': 'req-99435e2e-b910-4617-a90e-25bb418e1144'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:28.361411 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-99435e2e-b910-4617-a90e-25bb418e1144 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b08b03e04f394198bdf9a320a3a4064b', 'request_id': 'req-99435e2e-b910-4617-a90e-25bb418e1144'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:28.368608 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-99435e2e-b910-4617-a90e-25bb418e1144 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b08b03e04f394198bdf9a320a3a4064b', 'request_id': 'req-99435e2e-b910-4617-a90e-25bb418e1144'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:28.380391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-99435e2e-b910-4617-a90e-25bb418e1144 admin admin] The token cache is being invalidate because project b08b03e04f394198bdf9a320a3a4064b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:15:28.380680 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-99435e2e-b910-4617-a90e-25bb418e1144 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:28.381316 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-99435e2e-b910-4617-a90e-25bb418e1144 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:28.382896 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1207/2427] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:15:28 2022] DELETE /identity/v3/projects/b08b03e04f394198bdf9a320a3a4064b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:28.388500 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668600e-6d35-4d01-96d1-e05885954dc4 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:28.388796 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668600e-6d35-4d01-96d1-e05885954dc4 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:28.389083 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4668600e-6d35-4d01-96d1-e05885954dc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:28.396361 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=78,keystone:DELETE=20,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:28.408499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-4668600e-6d35-4d01-96d1-e05885954dc4 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:28.409726 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4668600e-6d35-4d01-96d1-e05885954dc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:28.469395 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1221/2428] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:15:28.477426 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5aeed-28b4-4d97-b402-0157437abfe4 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:28.477718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5aeed-28b4-4d97-b402-0157437abfe4 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:28.478001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3c5aeed-28b4-4d97-b402-0157437abfe4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:28.496578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f3c5aeed-28b4-4d97-b402-0157437abfe4 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:28.497790 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f3c5aeed-28b4-4d97-b402-0157437abfe4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:28.505727 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1208/2429] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:15:28.510846 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668600e-6d35-4d01-96d1-e05885954dc4 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:28.520535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4668600e-6d35-4d01-96d1-e05885954dc4 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:28.525088 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9b5c4-c0ba-476f-a8d4-25f060e493f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:28.525739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9b5c4-c0ba-476f-a8d4-25f060e493f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:28.526018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9b5c4-c0ba-476f-a8d4-25f060e493f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:28.526293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e9b5c4-c0ba-476f-a8d4-25f060e493f2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:28.527037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9b5c4-c0ba-476f-a8d4-25f060e493f2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:15:28.527343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9b5c4-c0ba-476f-a8d4-25f060e493f2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:28.528939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e9b5c4-c0ba-476f-a8d4-25f060e493f2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:28.540999 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1222/2430] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:15:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:15:28.545450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5aeed-28b4-4d97-b402-0157437abfe4 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:28.555035 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3c5aeed-28b4-4d97-b402-0157437abfe4 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:28.560274 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b30538-ebf6-461a-8a6e-ac6d221345a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:28.560926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b30538-ebf6-461a-8a6e-ac6d221345a7 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:28.561217 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b30538-ebf6-461a-8a6e-ac6d221345a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:28.561497 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73b30538-ebf6-461a-8a6e-ac6d221345a7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:28.562190 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b30538-ebf6-461a-8a6e-ac6d221345a7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:28.564109 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73b30538-ebf6-461a-8a6e-ac6d221345a7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:28.589882 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1209/2431] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:15:28 2022] POST /identity/v3/projects => generated 377 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:15:28.594137 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9b5c4-c0ba-476f-a8d4-25f060e493f2 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:28.604151 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e9b5c4-c0ba-476f-a8d4-25f060e493f2 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:28.659383 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bd10b-eac2-41aa-82e6-b32315b6df07 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:28.660050 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bd10b-eac2-41aa-82e6-b32315b6df07 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:28.660334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bd10b-eac2-41aa-82e6-b32315b6df07 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:28.660611 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e2bd10b-eac2-41aa-82e6-b32315b6df07 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:28.661310 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2bd10b-eac2-41aa-82e6-b32315b6df07 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:28.663244 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e2bd10b-eac2-41aa-82e6-b32315b6df07 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:28.684536 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1223/2432] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:15:28 2022] POST /identity/v3/users => generated 289 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:15:28.688611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b30538-ebf6-461a-8a6e-ac6d221345a7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:28.699048 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73b30538-ebf6-461a-8a6e-ac6d221345a7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:28.703926 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68eaf8a-62d6-43a5-aa71-ca399b10edf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:28.704599 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68eaf8a-62d6-43a5-aa71-ca399b10edf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:28.704887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68eaf8a-62d6-43a5-aa71-ca399b10edf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:28.705165 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d68eaf8a-62d6-43a5-aa71-ca399b10edf7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:28.706218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68eaf8a-62d6-43a5-aa71-ca399b10edf7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:28.709143 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d68eaf8a-62d6-43a5-aa71-ca399b10edf7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:28.723045 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1210/2433] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:15:28.727121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bd10b-eac2-41aa-82e6-b32315b6df07 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:28.737762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e2bd10b-eac2-41aa-82e6-b32315b6df07 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:28.742639 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87cbed8-a6e9-460b-9ecf-e802e3637195 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:28.743305 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cbed8-a6e9-460b-9ecf-e802e3637195 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:28.743596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cbed8-a6e9-460b-9ecf-e802e3637195 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:28.743874 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87cbed8-a6e9-460b-9ecf-e802e3637195 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:28.745269 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cbed8-a6e9-460b-9ecf-e802e3637195 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:28.746911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87cbed8-a6e9-460b-9ecf-e802e3637195 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:28.758270 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1224/2434] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:15:28 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:15:28.762168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68eaf8a-62d6-43a5-aa71-ca399b10edf7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:28.772785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d68eaf8a-62d6-43a5-aa71-ca399b10edf7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:28.778218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f51aa4-2fe1-4a90-89d9-41b98d111bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:28.779170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f51aa4-2fe1-4a90-89d9-41b98d111bb8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:28.779464 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f51aa4-2fe1-4a90-89d9-41b98d111bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:28.779750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f51aa4-2fe1-4a90-89d9-41b98d111bb8 admin admin] PATH_INFO: `/v3/projects/67d8660c31bf48e9a2c6a5edb2060bde/users/3c85d19cf42f4688bbc89a439d96ca69/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:28.795625 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f51aa4-2fe1-4a90-89d9-41b98d111bb8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=67d8660c31bf48e9a2c6a5edb2060bde, user_id=3c85d19cf42f4688bbc89a439d96ca69, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:28.798438 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f51aa4-2fe1-4a90-89d9-41b98d111bb8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:28.813690 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1211/2435] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:15:28 2022] PUT /identity/v3/projects/67d8660c31bf48e9a2c6a5edb2060bde/users/3c85d19cf42f4688bbc89a439d96ca69/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:28.818750 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e68636-464b-424e-8602-6394876cc504 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:28.819043 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e68636-464b-424e-8602-6394876cc504 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:28.819370 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6e68636-464b-424e-8602-6394876cc504 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:28.837144 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-f6e68636-464b-424e-8602-6394876cc504 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:28.839787 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f6e68636-464b-424e-8602-6394876cc504 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:28.901176 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1225/2436] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:15:28.910974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f51aa4-2fe1-4a90-89d9-41b98d111bb8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:28.912736 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a7f51aa4-2fe1-4a90-89d9-41b98d111bb8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:28.971788 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f51aa4-2fe1-4a90-89d9-41b98d111bb8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:28.975503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245816b-2221-4a59-a889-663af32439dc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:28.976085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245816b-2221-4a59-a889-663af32439dc service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:28.976367 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245816b-2221-4a59-a889-663af32439dc service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:28.976671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d245816b-2221-4a59-a889-663af32439dc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:28.986697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245816b-2221-4a59-a889-663af32439dc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:28.988107 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d245816b-2221-4a59-a889-663af32439dc service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:29.000189 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1212/2437] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:15:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:15:33.559059 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f17a1-76fe-423e-b5ac-7f99cab2f549 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:33.559654 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f17a1-76fe-423e-b5ac-7f99cab2f549 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:33.559806 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739f17a1-76fe-423e-b5ac-7f99cab2f549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:33.581889 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:33.591761 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-739f17a1-76fe-423e-b5ac-7f99cab2f549 None None] MFA Rules not processed for user `3c85d19cf42f4688bbc89a439d96ca69`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:33.593234 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-739f17a1-76fe-423e-b5ac-7f99cab2f549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:33.685739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1226/2438] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:33 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:15:33.686861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245816b-2221-4a59-a889-663af32439dc service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:33.698369 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d245816b-2221-4a59-a889-663af32439dc service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:33.703145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eace1b7-54c6-4e0d-be65-d6966c7be4c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:33.703796 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eace1b7-54c6-4e0d-be65-d6966c7be4c7 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:33.704084 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eace1b7-54c6-4e0d-be65-d6966c7be4c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:33.704360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eace1b7-54c6-4e0d-be65-d6966c7be4c7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:33.705049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eace1b7-54c6-4e0d-be65-d6966c7be4c7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:33.706989 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1eace1b7-54c6-4e0d-be65-d6966c7be4c7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:33.742075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1213/2439] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:15:33 2022] POST /identity/v3/projects => generated 379 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:15:33.746056 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f17a1-76fe-423e-b5ac-7f99cab2f549 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:33.758282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739f17a1-76fe-423e-b5ac-7f99cab2f549 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:33.824502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cea78-0782-404a-b06d-2f31c1039c93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:33.825226 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cea78-0782-404a-b06d-2f31c1039c93 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:33.825507 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cea78-0782-404a-b06d-2f31c1039c93 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:33.825786 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd0cea78-0782-404a-b06d-2f31c1039c93 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:33.826830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0cea78-0782-404a-b06d-2f31c1039c93 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:33.828803 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd0cea78-0782-404a-b06d-2f31c1039c93 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:33.860841 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1227/2440] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:15:33 2022] POST /identity/v3/users => generated 296 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:15:33.865493 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eace1b7-54c6-4e0d-be65-d6966c7be4c7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:33.880479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eace1b7-54c6-4e0d-be65-d6966c7be4c7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:33.887215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a098bf-08ed-4c8c-81c3-7f11831f7412 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:33.888069 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a098bf-08ed-4c8c-81c3-7f11831f7412 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:33.888419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a098bf-08ed-4c8c-81c3-7f11831f7412 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:33.888762 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a098bf-08ed-4c8c-81c3-7f11831f7412 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:33.889614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a098bf-08ed-4c8c-81c3-7f11831f7412 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:33.891415 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a098bf-08ed-4c8c-81c3-7f11831f7412 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:33.904122 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1214/2441] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:15:33.908977 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cea78-0782-404a-b06d-2f31c1039c93 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:33.920245 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd0cea78-0782-404a-b06d-2f31c1039c93 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:33.926126 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c407a-9433-482b-a91f-674ebaf7c7db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:33.927258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c407a-9433-482b-a91f-674ebaf7c7db admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:33.927670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c407a-9433-482b-a91f-674ebaf7c7db admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:33.927960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661c407a-9433-482b-a91f-674ebaf7c7db admin admin] PATH_INFO: `/v3/projects/f867bcd9ba0445bcbac83aaf3693ee6c/users/febf83e79bcc4f6f90e2c7a2aaf23c80/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:33.947718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c407a-9433-482b-a91f-674ebaf7c7db admin admin] RBAC: Authorizing `identity:create_grant(project_id=f867bcd9ba0445bcbac83aaf3693ee6c, user_id=febf83e79bcc4f6f90e2c7a2aaf23c80, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:33.951139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661c407a-9433-482b-a91f-674ebaf7c7db admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:33.969047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1228/2442] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:15:33 2022] PUT /identity/v3/projects/f867bcd9ba0445bcbac83aaf3693ee6c/users/febf83e79bcc4f6f90e2c7a2aaf23c80/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:15:33.973899 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a098bf-08ed-4c8c-81c3-7f11831f7412 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:33.984994 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a098bf-08ed-4c8c-81c3-7f11831f7412 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:33.990407 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:34.141686 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9a0c6-1b95-4a22-ae6b-5c7e438e5d2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:34.142852 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9a0c6-1b95-4a22-ae6b-5c7e438e5d2b admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:34.143400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9a0c6-1b95-4a22-ae6b-5c7e438e5d2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:34.143893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b9a0c6-1b95-4a22-ae6b-5c7e438e5d2b admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:34.144882 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9a0c6-1b95-4a22-ae6b-5c7e438e5d2b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:34.147382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b9a0c6-1b95-4a22-ae6b-5c7e438e5d2b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:34.168717 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1215/2443] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:15:33 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:15:34.173113 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c407a-9433-482b-a91f-674ebaf7c7db admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:34.202069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661c407a-9433-482b-a91f-674ebaf7c7db admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:34.207670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f493d9c6-6c17-4129-8685-c55975f06bb4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:34.209225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493d9c6-6c17-4129-8685-c55975f06bb4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:34.209502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493d9c6-6c17-4129-8685-c55975f06bb4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:34.209931 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f493d9c6-6c17-4129-8685-c55975f06bb4 admin admin] PATH_INFO: `/v3/projects/f867bcd9ba0445bcbac83aaf3693ee6c/users/febf83e79bcc4f6f90e2c7a2aaf23c80/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:34.219678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493d9c6-6c17-4129-8685-c55975f06bb4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f867bcd9ba0445bcbac83aaf3693ee6c, user_id=febf83e79bcc4f6f90e2c7a2aaf23c80, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:34.219678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f493d9c6-6c17-4129-8685-c55975f06bb4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:34.259274 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1229/2444] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:15:34 2022] PUT /identity/v3/projects/f867bcd9ba0445bcbac83aaf3693ee6c/users/febf83e79bcc4f6f90e2c7a2aaf23c80/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:15:35.507325 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9a0c6-1b95-4a22-ae6b-5c7e438e5d2b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:35.510154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-74b9a0c6-1b95-4a22-ae6b-5c7e438e5d2b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:35.524249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b9a0c6-1b95-4a22-ae6b-5c7e438e5d2b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:35.599011 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804f963-e023-484f-8c83-cdf2b79fe885 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:35.600341 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804f963-e023-484f-8c83-cdf2b79fe885 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:35.601885 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804f963-e023-484f-8c83-cdf2b79fe885 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:35.605750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804f963-e023-484f-8c83-cdf2b79fe885 admin admin] PATH_INFO: `/v3/users/d592d865f96944caa9cf1508af654332` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:35.609203 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3804f963-e023-484f-8c83-cdf2b79fe885 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d592d865f96944caa9cf1508af654332)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:35.611832 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3804f963-e023-484f-8c83-cdf2b79fe885 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:35.672643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3804f963-e023-484f-8c83-cdf2b79fe885 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd592d865f96944caa9cf1508af654332'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:35.673391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3804f963-e023-484f-8c83-cdf2b79fe885 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd592d865f96944caa9cf1508af654332'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:35.688274 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3804f963-e023-484f-8c83-cdf2b79fe885 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd592d865f96944caa9cf1508af654332'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:35.689348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3804f963-e023-484f-8c83-cdf2b79fe885 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd592d865f96944caa9cf1508af654332'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:35.713521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3804f963-e023-484f-8c83-cdf2b79fe885 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd592d865f96944caa9cf1508af654332'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:35.731645 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1216/2445] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:15:35 2022] DELETE /identity/v3/users/d592d865f96944caa9cf1508af654332 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:35.738001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f048bd0-122a-4109-8f03-0ecf1bd57c3b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:35.742149 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f048bd0-122a-4109-8f03-0ecf1bd57c3b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:35.743165 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f048bd0-122a-4109-8f03-0ecf1bd57c3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:35.767974 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9f048bd0-122a-4109-8f03-0ecf1bd57c3b None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:35.769576 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9f048bd0-122a-4109-8f03-0ecf1bd57c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:35.832695 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:35.850089 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1230/2446] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:15:35.861379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804f963-e023-484f-8c83-cdf2b79fe885 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:35.864626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3804f963-e023-484f-8c83-cdf2b79fe885 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:35.932443 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:35.975601 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804f963-e023-484f-8c83-cdf2b79fe885 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:35.980935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17e9da-64a8-4464-ba4f-d81d6465f3fc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:35.981613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17e9da-64a8-4464-ba4f-d81d6465f3fc service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:35.981898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17e9da-64a8-4464-ba4f-d81d6465f3fc service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:35.982180 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b17e9da-64a8-4464-ba4f-d81d6465f3fc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:35.995590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17e9da-64a8-4464-ba4f-d81d6465f3fc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:35.997039 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b17e9da-64a8-4464-ba4f-d81d6465f3fc service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:36.011150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1217/2447] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:15:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:15:36.436682 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f048bd0-122a-4109-8f03-0ecf1bd57c3b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:36.438647 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9f048bd0-122a-4109-8f03-0ecf1bd57c3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:36.451851 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f048bd0-122a-4109-8f03-0ecf1bd57c3b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:36.458568 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd010fcd-a412-495e-8193-3ca500ed94bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:36.460353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd010fcd-a412-495e-8193-3ca500ed94bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:36.460353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd010fcd-a412-495e-8193-3ca500ed94bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:36.460353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd010fcd-a412-495e-8193-3ca500ed94bf admin admin] PATH_INFO: `/v3/projects/bfae43c4a76d45a29563f347fd63e7e5` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:36.461519 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd010fcd-a412-495e-8193-3ca500ed94bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=bfae43c4a76d45a29563f347fd63e7e5)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:36.463673 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd010fcd-a412-495e-8193-3ca500ed94bf admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:36.547109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cd010fcd-a412-495e-8193-3ca500ed94bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bfae43c4a76d45a29563f347fd63e7e5', 'request_id': 'req-cd010fcd-a412-495e-8193-3ca500ed94bf'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:36.560722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cd010fcd-a412-495e-8193-3ca500ed94bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bfae43c4a76d45a29563f347fd63e7e5', 'request_id': 'req-cd010fcd-a412-495e-8193-3ca500ed94bf'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:36.568466 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cd010fcd-a412-495e-8193-3ca500ed94bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bfae43c4a76d45a29563f347fd63e7e5', 'request_id': 'req-cd010fcd-a412-495e-8193-3ca500ed94bf'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:36.584529 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cd010fcd-a412-495e-8193-3ca500ed94bf admin admin] The token cache is being invalidate because project bfae43c4a76d45a29563f347fd63e7e5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:15:36.585156 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cd010fcd-a412-495e-8193-3ca500ed94bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:36.586129 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-cd010fcd-a412-495e-8193-3ca500ed94bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:36.589214 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1231/2448] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:15:36 2022] DELETE /identity/v3/projects/bfae43c4a76d45a29563f347fd63e7e5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:36.593127 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17e9da-64a8-4464-ba4f-d81d6465f3fc service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:36.609937 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b17e9da-64a8-4464-ba4f-d81d6465f3fc service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:36.723694 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddb104f8-f344-4324-8b6f-5211b2ffc75c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:36.724862 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb104f8-f344-4324-8b6f-5211b2ffc75c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:36.725298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb104f8-f344-4324-8b6f-5211b2ffc75c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:36.725780 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb104f8-f344-4324-8b6f-5211b2ffc75c admin admin] PATH_INFO: `/v3/users/e8c3cf541f004c429a6d96812dc6a008` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:36.727236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb104f8-f344-4324-8b6f-5211b2ffc75c admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8c3cf541f004c429a6d96812dc6a008)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:36.730394 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddb104f8-f344-4324-8b6f-5211b2ffc75c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:36.877442 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ddb104f8-f344-4324-8b6f-5211b2ffc75c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8c3cf541f004c429a6d96812dc6a008'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:36.879530 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ddb104f8-f344-4324-8b6f-5211b2ffc75c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8c3cf541f004c429a6d96812dc6a008'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:36.908892 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ddb104f8-f344-4324-8b6f-5211b2ffc75c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8c3cf541f004c429a6d96812dc6a008'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:36.910142 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ddb104f8-f344-4324-8b6f-5211b2ffc75c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8c3cf541f004c429a6d96812dc6a008'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:36.958947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ddb104f8-f344-4324-8b6f-5211b2ffc75c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8c3cf541f004c429a6d96812dc6a008'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:36.997545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1218/2449] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:15:36 2022] DELETE /identity/v3/users/e8c3cf541f004c429a6d96812dc6a008 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:38.416818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd010fcd-a412-495e-8193-3ca500ed94bf admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:38.420338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-cd010fcd-a412-495e-8193-3ca500ed94bf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:38.436163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd010fcd-a412-495e-8193-3ca500ed94bf admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:38.620712 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daa18dbe-e7cd-440d-aef3-0a51d2ef4457 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:38.621626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa18dbe-e7cd-440d-aef3-0a51d2ef4457 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:38.621918 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa18dbe-e7cd-440d-aef3-0a51d2ef4457 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:38.622199 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daa18dbe-e7cd-440d-aef3-0a51d2ef4457 admin admin] PATH_INFO: `/v3/projects/4cdc2f0f7347413d900b484a98322970` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:38.623885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa18dbe-e7cd-440d-aef3-0a51d2ef4457 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cdc2f0f7347413d900b484a98322970)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:38.625850 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daa18dbe-e7cd-440d-aef3-0a51d2ef4457 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:38.753057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-daa18dbe-e7cd-440d-aef3-0a51d2ef4457 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cdc2f0f7347413d900b484a98322970', 'request_id': 'req-daa18dbe-e7cd-440d-aef3-0a51d2ef4457'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:38.768080 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-daa18dbe-e7cd-440d-aef3-0a51d2ef4457 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cdc2f0f7347413d900b484a98322970', 'request_id': 'req-daa18dbe-e7cd-440d-aef3-0a51d2ef4457'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:38.777249 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-daa18dbe-e7cd-440d-aef3-0a51d2ef4457 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cdc2f0f7347413d900b484a98322970', 'request_id': 'req-daa18dbe-e7cd-440d-aef3-0a51d2ef4457'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:38.811941 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-daa18dbe-e7cd-440d-aef3-0a51d2ef4457 admin admin] The token cache is being invalidate because project 4cdc2f0f7347413d900b484a98322970 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:15:38.812505 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-daa18dbe-e7cd-440d-aef3-0a51d2ef4457 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:38.813855 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-daa18dbe-e7cd-440d-aef3-0a51d2ef4457 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:38.817105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1232/2450] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:15:38 2022] DELETE /identity/v3/projects/4cdc2f0f7347413d900b484a98322970 => generated 0 bytes in 401 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:38.822859 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ad5fa-445e-4fec-9207-d89a79c6a75c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:38.823423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ad5fa-445e-4fec-9207-d89a79c6a75c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:38.823423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-167ad5fa-445e-4fec-9207-d89a79c6a75c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:38.869802 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-167ad5fa-445e-4fec-9207-d89a79c6a75c None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:38.871323 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-167ad5fa-445e-4fec-9207-d89a79c6a75c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:38.986583 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:39.038720 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1219/2451] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:15:39.049891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde38fc-bbf7-41a7-8fcd-2efb251ca82e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:39.049891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde38fc-bbf7-41a7-8fcd-2efb251ca82e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:39.049891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adde38fc-bbf7-41a7-8fcd-2efb251ca82e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:39.079208 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:39.101512 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-adde38fc-bbf7-41a7-8fcd-2efb251ca82e None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:39.108834 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-adde38fc-bbf7-41a7-8fcd-2efb251ca82e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:39.129417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1233/2452] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:15:39.133869 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ad5fa-445e-4fec-9207-d89a79c6a75c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:39.158719 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-167ad5fa-445e-4fec-9207-d89a79c6a75c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:39.172998 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f67493-9cee-4e86-816f-0101a0684e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:39.173295 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f67493-9cee-4e86-816f-0101a0684e29 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:39.173629 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f67493-9cee-4e86-816f-0101a0684e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:39.173896 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f67493-9cee-4e86-816f-0101a0684e29 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:39.175599 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f67493-9cee-4e86-816f-0101a0684e29 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:15:39.175599 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f67493-9cee-4e86-816f-0101a0684e29 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:39.178229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f67493-9cee-4e86-816f-0101a0684e29 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:39.216848 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1220/2453] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:15:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 14:15:39.217628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde38fc-bbf7-41a7-8fcd-2efb251ca82e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:39.232890 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adde38fc-bbf7-41a7-8fcd-2efb251ca82e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:39.243698 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e97ac-3cde-4871-9440-7ed50d316cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:39.245204 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e97ac-3cde-4871-9440-7ed50d316cc2 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:39.245324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e97ac-3cde-4871-9440-7ed50d316cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:39.245542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664e97ac-3cde-4871-9440-7ed50d316cc2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:39.246353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e97ac-3cde-4871-9440-7ed50d316cc2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:39.248314 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664e97ac-3cde-4871-9440-7ed50d316cc2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:39.324595 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1234/2454] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:15:39 2022] POST /identity/v3/projects => generated 385 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:15:39.328786 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f67493-9cee-4e86-816f-0101a0684e29 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:39.340559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f67493-9cee-4e86-816f-0101a0684e29 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:39.438447 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6270b-edeb-482c-9983-408385fb741c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:39.439494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6270b-edeb-482c-9983-408385fb741c admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:39.440164 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6270b-edeb-482c-9983-408385fb741c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:39.441342 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf6270b-edeb-482c-9983-408385fb741c admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:39.443085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6270b-edeb-482c-9983-408385fb741c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:39.445711 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cf6270b-edeb-482c-9983-408385fb741c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:39.485075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1221/2455] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:15:39 2022] POST /identity/v3/users => generated 293 bytes in 157 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:15:39.489359 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e97ac-3cde-4871-9440-7ed50d316cc2 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:39.505425 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664e97ac-3cde-4871-9440-7ed50d316cc2 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:39.515244 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75928e-30d0-4bf9-9ff0-623871610fd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:39.516090 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75928e-30d0-4bf9-9ff0-623871610fd4 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:39.516486 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75928e-30d0-4bf9-9ff0-623871610fd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:39.516883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed75928e-30d0-4bf9-9ff0-623871610fd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:39.517741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed75928e-30d0-4bf9-9ff0-623871610fd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:39.519875 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed75928e-30d0-4bf9-9ff0-623871610fd4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:39.540476 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1235/2456] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:15:39.548738 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6270b-edeb-482c-9983-408385fb741c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:39.571137 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cf6270b-edeb-482c-9983-408385fb741c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:39.584627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7a638d-dee4-4f90-b64d-6caebae42758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:39.585847 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a638d-dee4-4f90-b64d-6caebae42758 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:39.586339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a638d-dee4-4f90-b64d-6caebae42758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:39.586637 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7a638d-dee4-4f90-b64d-6caebae42758 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:39.589596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a638d-dee4-4f90-b64d-6caebae42758 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:39.603068 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7a638d-dee4-4f90-b64d-6caebae42758 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:39.626718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1222/2457] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:15:39 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:15:39.630970 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75928e-30d0-4bf9-9ff0-623871610fd4 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:39.643962 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed75928e-30d0-4bf9-9ff0-623871610fd4 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:39.651830 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc596017-dd5e-4ad3-8d96-caac9347de57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:39.653320 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc596017-dd5e-4ad3-8d96-caac9347de57 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:39.653832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc596017-dd5e-4ad3-8d96-caac9347de57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:39.654330 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc596017-dd5e-4ad3-8d96-caac9347de57 admin admin] PATH_INFO: `/v3/projects/7e5bf3092238482dbecae257ec9ec284/users/792982fd17e04aef8ea8904ab79cb977/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:39.675362 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc596017-dd5e-4ad3-8d96-caac9347de57 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7e5bf3092238482dbecae257ec9ec284, user_id=792982fd17e04aef8ea8904ab79cb977, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:39.680767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc596017-dd5e-4ad3-8d96-caac9347de57 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:39.721032 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609e266-84c1-4c59-96d0-21065d11a222 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:39.721250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609e266-84c1-4c59-96d0-21065d11a222 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:39.721641 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c609e266-84c1-4c59-96d0-21065d11a222 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:39.722627 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1236/2458] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:15:39 2022] PUT /identity/v3/projects/7e5bf3092238482dbecae257ec9ec284/users/792982fd17e04aef8ea8904ab79cb977/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 78 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:39.780264 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c609e266-84c1-4c59-96d0-21065d11a222 None None] MFA Rules not processed for user `792982fd17e04aef8ea8904ab79cb977`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:39.783324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c609e266-84c1-4c59-96d0-21065d11a222 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:39.967130 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1223/2459] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 245 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:15:39.968151 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc596017-dd5e-4ad3-8d96-caac9347de57 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:40.119822 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc596017-dd5e-4ad3-8d96-caac9347de57 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:40.148109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab8103-bc09-4948-980d-6f241a8d01ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:40.148804 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab8103-bc09-4948-980d-6f241a8d01ef admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:40.149089 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab8103-bc09-4948-980d-6f241a8d01ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:40.149373 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efab8103-bc09-4948-980d-6f241a8d01ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:40.150253 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab8103-bc09-4948-980d-6f241a8d01ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:40.153295 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efab8103-bc09-4948-980d-6f241a8d01ef admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:40.212541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1237/2460] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:15:39 2022] POST /identity/v3/projects => generated 383 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:15:40.214044 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609e266-84c1-4c59-96d0-21065d11a222 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:40.231853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c609e266-84c1-4c59-96d0-21065d11a222 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:40.363490 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:40.436078 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e87afd-2435-491a-b41b-6ad24ff623ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:40.436787 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e87afd-2435-491a-b41b-6ad24ff623ea admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:40.437072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e87afd-2435-491a-b41b-6ad24ff623ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:40.437346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87e87afd-2435-491a-b41b-6ad24ff623ea admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:40.438051 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e87afd-2435-491a-b41b-6ad24ff623ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:40.442095 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87e87afd-2435-491a-b41b-6ad24ff623ea admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:40.498940 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1224/2461] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:15:40 2022] POST /identity/v3/users => generated 298 bytes in 285 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:15:40.502673 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab8103-bc09-4948-980d-6f241a8d01ef admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:40.527528 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efab8103-bc09-4948-980d-6f241a8d01ef admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:40.533735 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e013590-17b0-425e-9870-829b9a127cc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:40.534553 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e013590-17b0-425e-9870-829b9a127cc7 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:40.535047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e013590-17b0-425e-9870-829b9a127cc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:40.535514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e013590-17b0-425e-9870-829b9a127cc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:40.536479 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e013590-17b0-425e-9870-829b9a127cc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:40.539015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e013590-17b0-425e-9870-829b9a127cc7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:40.553435 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1238/2462] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:15:40.557588 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e87afd-2435-491a-b41b-6ad24ff623ea admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:40.569592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87e87afd-2435-491a-b41b-6ad24ff623ea admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:40.574682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1928e-4474-4dfc-a4bf-4771ab9edcf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:40.575600 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1928e-4474-4dfc-a4bf-4771ab9edcf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:40.576260 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1928e-4474-4dfc-a4bf-4771ab9edcf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:40.576691 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a1928e-4474-4dfc-a4bf-4771ab9edcf4 admin admin] PATH_INFO: `/v3/projects/5de0164eebc34403a8581994fedbf300/users/7fbe6bb9f62d42efbc949a282d7a4df2/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:40.594917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1928e-4474-4dfc-a4bf-4771ab9edcf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5de0164eebc34403a8581994fedbf300, user_id=7fbe6bb9f62d42efbc949a282d7a4df2, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:40.598063 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a1928e-4474-4dfc-a4bf-4771ab9edcf4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:40.614726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1225/2463] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:15:40 2022] PUT /identity/v3/projects/5de0164eebc34403a8581994fedbf300/users/7fbe6bb9f62d42efbc949a282d7a4df2/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:40.618959 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e013590-17b0-425e-9870-829b9a127cc7 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:40.632666 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e013590-17b0-425e-9870-829b9a127cc7 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:40.740612 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71730fca-ec24-46dd-b084-0d3d34f72959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:40.741317 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71730fca-ec24-46dd-b084-0d3d34f72959 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:40.741610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71730fca-ec24-46dd-b084-0d3d34f72959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:40.741898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71730fca-ec24-46dd-b084-0d3d34f72959 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:40.742722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71730fca-ec24-46dd-b084-0d3d34f72959 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:40.744399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71730fca-ec24-46dd-b084-0d3d34f72959 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:40.794451 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1239/2464] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:15:40 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:15:40.797257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1928e-4474-4dfc-a4bf-4771ab9edcf4 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:40.808633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a1928e-4474-4dfc-a4bf-4771ab9edcf4 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:40.813549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f09acca-8a82-42ed-bd91-2e86f801e7a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:40.816592 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09acca-8a82-42ed-bd91-2e86f801e7a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:40.818462 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09acca-8a82-42ed-bd91-2e86f801e7a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:40.818776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f09acca-8a82-42ed-bd91-2e86f801e7a4 admin admin] PATH_INFO: `/v3/projects/5de0164eebc34403a8581994fedbf300/users/7fbe6bb9f62d42efbc949a282d7a4df2/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:40.821621 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09acca-8a82-42ed-bd91-2e86f801e7a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5de0164eebc34403a8581994fedbf300, user_id=7fbe6bb9f62d42efbc949a282d7a4df2, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:40.826598 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f09acca-8a82-42ed-bd91-2e86f801e7a4 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:40.837268 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1226/2465] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:15:40 2022] PUT /identity/v3/projects/5de0164eebc34403a8581994fedbf300/users/7fbe6bb9f62d42efbc949a282d7a4df2/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:40.847926 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9cb47-4319-4ffd-b1f9-558a24e764f2 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:40.848615 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9cb47-4319-4ffd-b1f9-558a24e764f2 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:40.848961 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87c9cb47-4319-4ffd-b1f9-558a24e764f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:40.896336 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4d50b-089e-4f76-94de-00efbf56e578 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:40.896573 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4d50b-089e-4f76-94de-00efbf56e578 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:40.896852 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86c4d50b-089e-4f76-94de-00efbf56e578 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:40.909039 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:40.916706 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-87c9cb47-4319-4ffd-b1f9-558a24e764f2 None None] MFA Rules not processed for user `7fbe6bb9f62d42efbc949a282d7a4df2`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:40.918188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-87c9cb47-4319-4ffd-b1f9-558a24e764f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:40.943368 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-86c4d50b-089e-4f76-94de-00efbf56e578 None None] MFA Rules not processed for user `febf83e79bcc4f6f90e2c7a2aaf23c80`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:40.944919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-86c4d50b-089e-4f76-94de-00efbf56e578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:41.048416 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1240/2466] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:40 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:15:41.066900 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9cb47-4319-4ffd-b1f9-558a24e764f2 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:41.068550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-87c9cb47-4319-4ffd-b1f9-558a24e764f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:41.085124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1227/2467] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:40 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:15:41.102492 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4d50b-089e-4f76-94de-00efbf56e578 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:41.104652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-86c4d50b-089e-4f76-94de-00efbf56e578 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:41.182928 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87c9cb47-4319-4ffd-b1f9-558a24e764f2 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:41.194206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b52b9e-c4eb-4a9d-8f43-db61e2d10d1e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:41.195071 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b52b9e-c4eb-4a9d-8f43-db61e2d10d1e service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:41.195371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b52b9e-c4eb-4a9d-8f43-db61e2d10d1e service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:41.195660 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3b52b9e-c4eb-4a9d-8f43-db61e2d10d1e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:41.270985 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b52b9e-c4eb-4a9d-8f43-db61e2d10d1e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:41.272530 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3b52b9e-c4eb-4a9d-8f43-db61e2d10d1e service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:41.285850 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1241/2468] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:15:41 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 219 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:15:41.368269 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:41.397652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86c4d50b-089e-4f76-94de-00efbf56e578 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:41.408528 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d32050-3597-461f-8e74-dbd063ff6c8c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:41.409202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d32050-3597-461f-8e74-dbd063ff6c8c service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:41.409487 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d32050-3597-461f-8e74-dbd063ff6c8c service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:41.409819 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d32050-3597-461f-8e74-dbd063ff6c8c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:41.411654 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b1d32050-3597-461f-8e74-dbd063ff6c8c service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:41.503398 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d32050-3597-461f-8e74-dbd063ff6c8c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:41.506232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d32050-3597-461f-8e74-dbd063ff6c8c service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:41.523867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1228/2469] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:15:41 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 422 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:15:45.769174 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b52b9e-c4eb-4a9d-8f43-db61e2d10d1e service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:45.778821 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3b52b9e-c4eb-4a9d-8f43-db61e2d10d1e service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:45.782954 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4005f4e8-5c4a-4949-9994-b871046ab8b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:45.783672 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005f4e8-5c4a-4949-9994-b871046ab8b4 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:45.783940 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005f4e8-5c4a-4949-9994-b871046ab8b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:45.784229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4005f4e8-5c4a-4949-9994-b871046ab8b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:45.795510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005f4e8-5c4a-4949-9994-b871046ab8b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:45.796976 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4005f4e8-5c4a-4949-9994-b871046ab8b4 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:45.818973 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1242/2470] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:15:45 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:15:50.894426 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d32050-3597-461f-8e74-dbd063ff6c8c service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:50.967060 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d32050-3597-461f-8e74-dbd063ff6c8c service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:50.971642 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749eaee-2405-43d5-8469-45c6a8fd6120 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:50.972515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749eaee-2405-43d5-8469-45c6a8fd6120 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:50.972804 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749eaee-2405-43d5-8469-45c6a8fd6120 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:50.973081 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4749eaee-2405-43d5-8469-45c6a8fd6120 admin admin] PATH_INFO: `/v3/users/792982fd17e04aef8ea8904ab79cb977` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:50.974333 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749eaee-2405-43d5-8469-45c6a8fd6120 admin admin] RBAC: Authorizing `identity:delete_user(user_id=792982fd17e04aef8ea8904ab79cb977)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:50.976267 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4749eaee-2405-43d5-8469-45c6a8fd6120 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:51.025747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4749eaee-2405-43d5-8469-45c6a8fd6120 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '792982fd17e04aef8ea8904ab79cb977'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:51.026454 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4749eaee-2405-43d5-8469-45c6a8fd6120 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '792982fd17e04aef8ea8904ab79cb977'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:51.033874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4749eaee-2405-43d5-8469-45c6a8fd6120 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '792982fd17e04aef8ea8904ab79cb977'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:51.034586 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4749eaee-2405-43d5-8469-45c6a8fd6120 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '792982fd17e04aef8ea8904ab79cb977'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:51.052335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4749eaee-2405-43d5-8469-45c6a8fd6120 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '792982fd17e04aef8ea8904ab79cb977'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:51.074371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1229/2471] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:15:50 2022] DELETE /identity/v3/users/792982fd17e04aef8ea8904ab79cb977 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:51.078563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d92b14-c560-497f-8c62-39ccb21394b3 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:51.079308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d92b14-c560-497f-8c62-39ccb21394b3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:51.079308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d92b14-c560-497f-8c62-39ccb21394b3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:51.097195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e7d92b14-c560-497f-8c62-39ccb21394b3 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:51.098448 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e7d92b14-c560-497f-8c62-39ccb21394b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:51.137125 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:51.164496 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1243/2472] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:15:51.176819 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749eaee-2405-43d5-8469-45c6a8fd6120 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:51.178486 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4749eaee-2405-43d5-8469-45c6a8fd6120 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:51.192540 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:51.245114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4749eaee-2405-43d5-8469-45c6a8fd6120 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:51.249030 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935ee1d-447f-4edd-b80a-0f589188c356 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:51.249581 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935ee1d-447f-4edd-b80a-0f589188c356 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:51.249869 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935ee1d-447f-4edd-b80a-0f589188c356 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:51.250151 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5935ee1d-447f-4edd-b80a-0f589188c356 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:51.260292 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935ee1d-447f-4edd-b80a-0f589188c356 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:51.261720 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5935ee1d-447f-4edd-b80a-0f589188c356 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:51.280334 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1230/2473] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:15:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:15:51.576082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d92b14-c560-497f-8c62-39ccb21394b3 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:51.577839 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e7d92b14-c560-497f-8c62-39ccb21394b3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:51.591321 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d92b14-c560-497f-8c62-39ccb21394b3 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:51.596236 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a65c0f-da5b-4b0d-94b9-81f187a16b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:51.597306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a65c0f-da5b-4b0d-94b9-81f187a16b73 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:51.597587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a65c0f-da5b-4b0d-94b9-81f187a16b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:51.597860 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29a65c0f-da5b-4b0d-94b9-81f187a16b73 admin admin] PATH_INFO: `/v3/projects/7e5bf3092238482dbecae257ec9ec284` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:51.599201 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a65c0f-da5b-4b0d-94b9-81f187a16b73 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7e5bf3092238482dbecae257ec9ec284)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:51.601146 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29a65c0f-da5b-4b0d-94b9-81f187a16b73 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:51.652110 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-29a65c0f-da5b-4b0d-94b9-81f187a16b73 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7e5bf3092238482dbecae257ec9ec284', 'request_id': 'req-29a65c0f-da5b-4b0d-94b9-81f187a16b73'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:51.661265 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-29a65c0f-da5b-4b0d-94b9-81f187a16b73 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7e5bf3092238482dbecae257ec9ec284', 'request_id': 'req-29a65c0f-da5b-4b0d-94b9-81f187a16b73'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:51.672269 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-29a65c0f-da5b-4b0d-94b9-81f187a16b73 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7e5bf3092238482dbecae257ec9ec284', 'request_id': 'req-29a65c0f-da5b-4b0d-94b9-81f187a16b73'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:51.684527 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-29a65c0f-da5b-4b0d-94b9-81f187a16b73 admin admin] The token cache is being invalidate because project 7e5bf3092238482dbecae257ec9ec284 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:15:51.684817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-29a65c0f-da5b-4b0d-94b9-81f187a16b73 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:51.685611 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-29a65c0f-da5b-4b0d-94b9-81f187a16b73 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:51.687256 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1244/2474] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:15:51 2022] DELETE /identity/v3/projects/7e5bf3092238482dbecae257ec9ec284 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:51.691613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935ee1d-447f-4edd-b80a-0f589188c356 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:51.702220 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5935ee1d-447f-4edd-b80a-0f589188c356 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:51.768552 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8181da8d-71ab-419b-af99-691bdcd1539f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:51.769495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181da8d-71ab-419b-af99-691bdcd1539f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:51.769777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181da8d-71ab-419b-af99-691bdcd1539f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:51.770053 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8181da8d-71ab-419b-af99-691bdcd1539f admin admin] PATH_INFO: `/v3/users/7fbe6bb9f62d42efbc949a282d7a4df2` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:51.771416 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181da8d-71ab-419b-af99-691bdcd1539f admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fbe6bb9f62d42efbc949a282d7a4df2)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:51.773363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8181da8d-71ab-419b-af99-691bdcd1539f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:51.824140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8181da8d-71ab-419b-af99-691bdcd1539f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fbe6bb9f62d42efbc949a282d7a4df2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:51.824854 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8181da8d-71ab-419b-af99-691bdcd1539f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fbe6bb9f62d42efbc949a282d7a4df2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:51.832187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8181da8d-71ab-419b-af99-691bdcd1539f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fbe6bb9f62d42efbc949a282d7a4df2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:51.832831 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8181da8d-71ab-419b-af99-691bdcd1539f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fbe6bb9f62d42efbc949a282d7a4df2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:51.846496 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8181da8d-71ab-419b-af99-691bdcd1539f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fbe6bb9f62d42efbc949a282d7a4df2'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:51.859385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1231/2475] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:15:51 2022] DELETE /identity/v3/users/7fbe6bb9f62d42efbc949a282d7a4df2 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:52.154639 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a65c0f-da5b-4b0d-94b9-81f187a16b73 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:52.156640 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-29a65c0f-da5b-4b0d-94b9-81f187a16b73 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:52.175766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29a65c0f-da5b-4b0d-94b9-81f187a16b73 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:52.233381 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0b5076d-32ef-409a-9948-38d66d801d7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:52.234170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5076d-32ef-409a-9948-38d66d801d7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:52.234494 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5076d-32ef-409a-9948-38d66d801d7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:52.234778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0b5076d-32ef-409a-9948-38d66d801d7a admin admin] PATH_INFO: `/v3/projects/5de0164eebc34403a8581994fedbf300` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:52.235959 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b5076d-32ef-409a-9948-38d66d801d7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=5de0164eebc34403a8581994fedbf300)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:52.237875 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0b5076d-32ef-409a-9948-38d66d801d7a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:52.284487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a0b5076d-32ef-409a-9948-38d66d801d7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5de0164eebc34403a8581994fedbf300', 'request_id': 'req-a0b5076d-32ef-409a-9948-38d66d801d7a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:52.294409 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a0b5076d-32ef-409a-9948-38d66d801d7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5de0164eebc34403a8581994fedbf300', 'request_id': 'req-a0b5076d-32ef-409a-9948-38d66d801d7a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:52.301790 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a0b5076d-32ef-409a-9948-38d66d801d7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5de0164eebc34403a8581994fedbf300', 'request_id': 'req-a0b5076d-32ef-409a-9948-38d66d801d7a'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:52.303049 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:52.318205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a0b5076d-32ef-409a-9948-38d66d801d7a admin admin] The token cache is being invalidate because project 5de0164eebc34403a8581994fedbf300 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:15:52.318670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a0b5076d-32ef-409a-9948-38d66d801d7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:52.319371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-a0b5076d-32ef-409a-9948-38d66d801d7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:15:52.321036 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1245/2476] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:15:52 2022] DELETE /identity/v3/projects/5de0164eebc34403a8581994fedbf300 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:52.327630 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fae18d-4eb5-450d-b879-e70057b0d51c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:52.328139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fae18d-4eb5-450d-b879-e70057b0d51c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:52.328250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2fae18d-4eb5-450d-b879-e70057b0d51c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:52.346393 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f2fae18d-4eb5-450d-b879-e70057b0d51c None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:52.347661 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f2fae18d-4eb5-450d-b879-e70057b0d51c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:52.374123 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:52.430034 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1232/2477] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:15:52.439092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48b958-d41f-4f79-b33e-9588c99e5e43 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:52.439386 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48b958-d41f-4f79-b33e-9588c99e5e43 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:52.439664 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf48b958-d41f-4f79-b33e-9588c99e5e43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:52.462374 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-cf48b958-d41f-4f79-b33e-9588c99e5e43 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:52.463528 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-cf48b958-d41f-4f79-b33e-9588c99e5e43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:52.471851 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1246/2478] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:15:52.476010 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fae18d-4eb5-450d-b879-e70057b0d51c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:52.485758 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2fae18d-4eb5-450d-b879-e70057b0d51c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:52.490334 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f15a44-8b79-409d-ae2a-a9d933cdbeb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:52.490985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f15a44-8b79-409d-ae2a-a9d933cdbeb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:52.491279 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f15a44-8b79-409d-ae2a-a9d933cdbeb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:52.491560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35f15a44-8b79-409d-ae2a-a9d933cdbeb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:52.492294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f15a44-8b79-409d-ae2a-a9d933cdbeb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:15:52.492609 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f15a44-8b79-409d-ae2a-a9d933cdbeb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:52.494770 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35f15a44-8b79-409d-ae2a-a9d933cdbeb3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:52.506587 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1233/2479] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:15:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:15:52.511954 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48b958-d41f-4f79-b33e-9588c99e5e43 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:52.521402 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf48b958-d41f-4f79-b33e-9588c99e5e43 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:52.526314 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4a647-d034-4946-902b-d1c7beee50fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:52.526985 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4a647-d034-4946-902b-d1c7beee50fe admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:52.527274 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4a647-d034-4946-902b-d1c7beee50fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:52.527555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b4a647-d034-4946-902b-d1c7beee50fe admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:52.528277 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4a647-d034-4946-902b-d1c7beee50fe admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:52.530353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b4a647-d034-4946-902b-d1c7beee50fe admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:52.568051 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1247/2480] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:15:52 2022] POST /identity/v3/projects => generated 383 bytes in 53 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:15:52.570590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f15a44-8b79-409d-ae2a-a9d933cdbeb3 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:52.581484 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35f15a44-8b79-409d-ae2a-a9d933cdbeb3 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:52.642296 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce08e6-9b6c-4f17-bf1c-2b2b235b9ff5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:52.643024 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce08e6-9b6c-4f17-bf1c-2b2b235b9ff5 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:52.643323 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce08e6-9b6c-4f17-bf1c-2b2b235b9ff5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:52.643603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ce08e6-9b6c-4f17-bf1c-2b2b235b9ff5 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:52.644406 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce08e6-9b6c-4f17-bf1c-2b2b235b9ff5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:52.646387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ce08e6-9b6c-4f17-bf1c-2b2b235b9ff5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:52.667770 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1234/2481] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:15:52 2022] POST /identity/v3/users => generated 292 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:15:52.671949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4a647-d034-4946-902b-d1c7beee50fe admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:52.683462 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b4a647-d034-4946-902b-d1c7beee50fe admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:52.688132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8fda7-d7cc-435e-baee-ae71556518d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:52.688806 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8fda7-d7cc-435e-baee-ae71556518d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:52.689180 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8fda7-d7cc-435e-baee-ae71556518d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:52.689466 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f8fda7-d7cc-435e-baee-ae71556518d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:52.690206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8fda7-d7cc-435e-baee-ae71556518d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:52.691902 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f8fda7-d7cc-435e-baee-ae71556518d1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:52.703302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1248/2482] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:15:52.707504 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce08e6-9b6c-4f17-bf1c-2b2b235b9ff5 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:52.717877 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ce08e6-9b6c-4f17-bf1c-2b2b235b9ff5 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:52.723944 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51a72fb3-0855-419a-a098-dbace54b8d86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:52.724615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a72fb3-0855-419a-a098-dbace54b8d86 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:52.724895 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a72fb3-0855-419a-a098-dbace54b8d86 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:52.725166 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51a72fb3-0855-419a-a098-dbace54b8d86 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:52.725904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a72fb3-0855-419a-a098-dbace54b8d86 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:52.727526 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51a72fb3-0855-419a-a098-dbace54b8d86 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:52.738463 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1235/2483] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:15:52 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:15:52.742765 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8fda7-d7cc-435e-baee-ae71556518d1 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:52.753618 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f8fda7-d7cc-435e-baee-ae71556518d1 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:52.759217 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744a98b-618d-48b2-8938-f4fd048d65ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:52.760112 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744a98b-618d-48b2-8938-f4fd048d65ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:52.760399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744a98b-618d-48b2-8938-f4fd048d65ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:52.760680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a744a98b-618d-48b2-8938-f4fd048d65ce admin admin] PATH_INFO: `/v3/projects/db8a8aee07ce44359359190b21aa5f82/users/b74c5a362206457ca5475eab90e3fda5/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:52.776158 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744a98b-618d-48b2-8938-f4fd048d65ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=db8a8aee07ce44359359190b21aa5f82, user_id=b74c5a362206457ca5475eab90e3fda5, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:52.779063 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a744a98b-618d-48b2-8938-f4fd048d65ce admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:52.797033 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1249/2484] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:15:52 2022] PUT /identity/v3/projects/db8a8aee07ce44359359190b21aa5f82/users/b74c5a362206457ca5475eab90e3fda5/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:52.811808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a72b7-1496-455d-ab99-51262d715df2 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:52.812095 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a72b7-1496-455d-ab99-51262d715df2 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:52.812364 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee5a72b7-1496-455d-ab99-51262d715df2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:52.843563 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ee5a72b7-1496-455d-ab99-51262d715df2 None None] MFA Rules not processed for user `b74c5a362206457ca5475eab90e3fda5`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:52.844795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ee5a72b7-1496-455d-ab99-51262d715df2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:52.927488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1236/2485] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:52 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:15:52.932994 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744a98b-618d-48b2-8938-f4fd048d65ce admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:53.002571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a744a98b-618d-48b2-8938-f4fd048d65ce admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:53.007266 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db40408-f873-4657-a9e1-73e4057a22df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:53.007946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db40408-f873-4657-a9e1-73e4057a22df admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:53.008239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db40408-f873-4657-a9e1-73e4057a22df admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:53.008513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3db40408-f873-4657-a9e1-73e4057a22df admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:53.009291 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db40408-f873-4657-a9e1-73e4057a22df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:53.012481 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3db40408-f873-4657-a9e1-73e4057a22df admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:53.045580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1250/2486] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:15:52 2022] POST /identity/v3/projects => generated 385 bytes in 113 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:15:53.049686 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a72b7-1496-455d-ab99-51262d715df2 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:53.060865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee5a72b7-1496-455d-ab99-51262d715df2 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:53.086758 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:53.121310 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb8698-564e-48fa-acd0-161ec3981560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:53.121982 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb8698-564e-48fa-acd0-161ec3981560 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:53.122265 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb8698-564e-48fa-acd0-161ec3981560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:53.122570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eb8698-564e-48fa-acd0-161ec3981560 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:53.123263 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb8698-564e-48fa-acd0-161ec3981560 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:53.125168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eb8698-564e-48fa-acd0-161ec3981560 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:53.149167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1237/2487] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:15:53 2022] POST /identity/v3/users => generated 293 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:15:53.153693 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db40408-f873-4657-a9e1-73e4057a22df admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:53.164704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3db40408-f873-4657-a9e1-73e4057a22df admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:53.170447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9abbf8-7a16-4f04-866e-6ef5fb944ac1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:53.171206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9abbf8-7a16-4f04-866e-6ef5fb944ac1 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:53.171504 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9abbf8-7a16-4f04-866e-6ef5fb944ac1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:53.171788 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b9abbf8-7a16-4f04-866e-6ef5fb944ac1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:53.172566 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9abbf8-7a16-4f04-866e-6ef5fb944ac1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:53.174185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b9abbf8-7a16-4f04-866e-6ef5fb944ac1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:53.186102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1251/2488] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:15:53.190408 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb8698-564e-48fa-acd0-161ec3981560 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:53.200750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eb8698-564e-48fa-acd0-161ec3981560 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:53.205436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6a8ccef-27cd-4b5b-a132-4b99eb4672ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:53.206114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8ccef-27cd-4b5b-a132-4b99eb4672ce admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:53.206446 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8ccef-27cd-4b5b-a132-4b99eb4672ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:53.206730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a8ccef-27cd-4b5b-a132-4b99eb4672ce admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:53.207509 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8ccef-27cd-4b5b-a132-4b99eb4672ce admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:53.209215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6a8ccef-27cd-4b5b-a132-4b99eb4672ce admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:53.220752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1238/2489] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:15:53.225013 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9abbf8-7a16-4f04-866e-6ef5fb944ac1 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:53.235728 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b9abbf8-7a16-4f04-866e-6ef5fb944ac1 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:53.240348 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b462fa-393f-4c95-9a6f-89a22354f7fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:53.241237 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b462fa-393f-4c95-9a6f-89a22354f7fb admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:53.241526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b462fa-393f-4c95-9a6f-89a22354f7fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:53.241808 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b462fa-393f-4c95-9a6f-89a22354f7fb admin admin] PATH_INFO: `/v3/projects/c245dbdd7a9741ca8d0f72524c37113e/users/7503ade196d942acaef4c43e08da58c2/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:53.256366 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b462fa-393f-4c95-9a6f-89a22354f7fb admin admin] RBAC: Authorizing `identity:create_grant(project_id=c245dbdd7a9741ca8d0f72524c37113e, user_id=7503ade196d942acaef4c43e08da58c2, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:53.259012 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b462fa-393f-4c95-9a6f-89a22354f7fb admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:53.271699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1252/2490] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:15:53 2022] PUT /identity/v3/projects/c245dbdd7a9741ca8d0f72524c37113e/users/7503ade196d942acaef4c43e08da58c2/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:53.282802 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d577e-b8f7-4a1f-b8c5-869b1fd645ab None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:53.283095 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d577e-b8f7-4a1f-b8c5-869b1fd645ab None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:53.283376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-648d577e-b8f7-4a1f-b8c5-869b1fd645ab None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:53.313403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-648d577e-b8f7-4a1f-b8c5-869b1fd645ab None None] MFA Rules not processed for user `7503ade196d942acaef4c43e08da58c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:53.314678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-648d577e-b8f7-4a1f-b8c5-869b1fd645ab None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:53.380683 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1239/2491] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:53 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:15:53.386321 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b462fa-393f-4c95-9a6f-89a22354f7fb admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:53.432297 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:53.452616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b462fa-393f-4c95-9a6f-89a22354f7fb admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:53.457247 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad4a63-0478-41db-812f-d4a83dcb579e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:53.457902 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad4a63-0478-41db-812f-d4a83dcb579e admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:53.458182 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad4a63-0478-41db-812f-d4a83dcb579e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:53.458486 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ad4a63-0478-41db-812f-d4a83dcb579e admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:53.459171 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad4a63-0478-41db-812f-d4a83dcb579e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:53.461053 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ad4a63-0478-41db-812f-d4a83dcb579e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:53.489822 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1253/2492] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:15:53 2022] POST /identity/v3/projects => generated 385 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:15:53.491150 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d577e-b8f7-4a1f-b8c5-869b1fd645ab None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:53.502515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-648d577e-b8f7-4a1f-b8c5-869b1fd645ab None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:53.563508 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e087d56-e4ef-4927-995a-d52bc5dc0012 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:53.564093 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e087d56-e4ef-4927-995a-d52bc5dc0012 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:53.564378 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e087d56-e4ef-4927-995a-d52bc5dc0012 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:53.564653 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e087d56-e4ef-4927-995a-d52bc5dc0012 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:53.565344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e087d56-e4ef-4927-995a-d52bc5dc0012 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:53.567372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e087d56-e4ef-4927-995a-d52bc5dc0012 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:53.593058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1240/2493] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:15:53 2022] POST /identity/v3/users => generated 299 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:15:53.594189 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad4a63-0478-41db-812f-d4a83dcb579e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:53.605675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ad4a63-0478-41db-812f-d4a83dcb579e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:53.611836 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434a2cb-bc90-44e8-b974-c9841b130434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:53.612991 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434a2cb-bc90-44e8-b974-c9841b130434 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:53.613309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434a2cb-bc90-44e8-b974-c9841b130434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:53.613686 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6434a2cb-bc90-44e8-b974-c9841b130434 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:53.614839 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434a2cb-bc90-44e8-b974-c9841b130434 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:53.617570 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6434a2cb-bc90-44e8-b974-c9841b130434 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:53.632554 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1254/2494] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:15:53.637197 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e087d56-e4ef-4927-995a-d52bc5dc0012 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:53.651078 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e087d56-e4ef-4927-995a-d52bc5dc0012 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:53.656024 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152e178-08e5-4175-8851-c4a965a6eb42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:53.656983 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152e178-08e5-4175-8851-c4a965a6eb42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:53.657818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152e178-08e5-4175-8851-c4a965a6eb42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:53.657818 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b152e178-08e5-4175-8851-c4a965a6eb42 admin admin] PATH_INFO: `/v3/projects/1df755d98ad94dd2a0942b8fa8f01923/users/5a85b1c3088e40f4b3a9bf3d63346a06/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:53.673168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b152e178-08e5-4175-8851-c4a965a6eb42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1df755d98ad94dd2a0942b8fa8f01923, user_id=5a85b1c3088e40f4b3a9bf3d63346a06, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:53.676115 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b152e178-08e5-4175-8851-c4a965a6eb42 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:53.685914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:53.691059 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1241/2495] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:15:53 2022] PUT /identity/v3/projects/1df755d98ad94dd2a0942b8fa8f01923/users/5a85b1c3088e40f4b3a9bf3d63346a06/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:15:53.695058 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434a2cb-bc90-44e8-b974-c9841b130434 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:53.705471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6434a2cb-bc90-44e8-b974-c9841b130434 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:53.762720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a88977-18ce-4b54-af82-0642ca4d7870 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:53.763414 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a88977-18ce-4b54-af82-0642ca4d7870 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:53.763685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a88977-18ce-4b54-af82-0642ca4d7870 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:53.763968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a88977-18ce-4b54-af82-0642ca4d7870 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:53.764695 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a88977-18ce-4b54-af82-0642ca4d7870 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:53.766324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a88977-18ce-4b54-af82-0642ca4d7870 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:53.777202 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1255/2496] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:15:53 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:15:53.781380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152e178-08e5-4175-8851-c4a965a6eb42 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:53.791435 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b152e178-08e5-4175-8851-c4a965a6eb42 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:53.796389 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa44d3a-9d0d-48c6-8b71-d35dbafdd810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:53.797279 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa44d3a-9d0d-48c6-8b71-d35dbafdd810 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:53.797565 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa44d3a-9d0d-48c6-8b71-d35dbafdd810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:53.797844 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfa44d3a-9d0d-48c6-8b71-d35dbafdd810 admin admin] PATH_INFO: `/v3/projects/1df755d98ad94dd2a0942b8fa8f01923/users/5a85b1c3088e40f4b3a9bf3d63346a06/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:53.800130 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa44d3a-9d0d-48c6-8b71-d35dbafdd810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1df755d98ad94dd2a0942b8fa8f01923, user_id=5a85b1c3088e40f4b3a9bf3d63346a06, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:53.802799 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfa44d3a-9d0d-48c6-8b71-d35dbafdd810 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:53.813687 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1242/2497] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:15:53 2022] PUT /identity/v3/projects/1df755d98ad94dd2a0942b8fa8f01923/users/5a85b1c3088e40f4b3a9bf3d63346a06/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:15:53.825326 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dd769-fcf6-4c7e-99f5-9c696030b70e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:53.825623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dd769-fcf6-4c7e-99f5-9c696030b70e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:53.825907 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20dd769-fcf6-4c7e-99f5-9c696030b70e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:53.854802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a20dd769-fcf6-4c7e-99f5-9c696030b70e None None] MFA Rules not processed for user `5a85b1c3088e40f4b3a9bf3d63346a06`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:15:53.856263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a20dd769-fcf6-4c7e-99f5-9c696030b70e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:53.929304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1256/2498] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:15:53 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:15:53.941403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa44d3a-9d0d-48c6-8b71-d35dbafdd810 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:53.943168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cfa44d3a-9d0d-48c6-8b71-d35dbafdd810 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:15:54.004410 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfa44d3a-9d0d-48c6-8b71-d35dbafdd810 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:54.010266 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271615e-81f6-4089-aece-be0bfbe9d270 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:54.010517 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271615e-81f6-4089-aece-be0bfbe9d270 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:54.010913 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271615e-81f6-4089-aece-be0bfbe9d270 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:54.011074 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271615e-81f6-4089-aece-be0bfbe9d270 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:54.070487 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271615e-81f6-4089-aece-be0bfbe9d270 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:54.073422 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271615e-81f6-4089-aece-be0bfbe9d270 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:54.083542 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1243/2499] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:15:53 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:15:55.599943 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20dd769-fcf6-4c7e-99f5-9c696030b70e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:55.610393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20dd769-fcf6-4c7e-99f5-9c696030b70e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:55.614014 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d2666-0eaf-457c-9b03-1ad57705e6cf service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:55.614706 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d2666-0eaf-457c-9b03-1ad57705e6cf service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:55.614998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d2666-0eaf-457c-9b03-1ad57705e6cf service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:55.615282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5d2666-0eaf-457c-9b03-1ad57705e6cf service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:55.649211 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:15:55.676777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d2666-0eaf-457c-9b03-1ad57705e6cf service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:55.678221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c5d2666-0eaf-457c-9b03-1ad57705e6cf service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:55.689348 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1257/2500] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:15:55 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:15:56.275424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271615e-81f6-4089-aece-be0bfbe9d270 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:15:56.283914 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271615e-81f6-4089-aece-be0bfbe9d270 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:15:56.287545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ef1fdf-5220-485f-bbb9-54d2e2f623bb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:15:56.288182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef1fdf-5220-485f-bbb9-54d2e2f623bb service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:15:56.288466 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef1fdf-5220-485f-bbb9-54d2e2f623bb service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:15:56.288744 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ef1fdf-5220-485f-bbb9-54d2e2f623bb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:15:56.298326 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef1fdf-5220-485f-bbb9-54d2e2f623bb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:15:56.299745 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82ef1fdf-5220-485f-bbb9-54d2e2f623bb service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:15:56.310101 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1244/2501] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:15:56 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:16:01.182938 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d2666-0eaf-457c-9b03-1ad57705e6cf service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:16:01.244565 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c5d2666-0eaf-457c-9b03-1ad57705e6cf service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:16:01.248981 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4632c5c-b04c-4fea-af34-adb309454fbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:16:01.249854 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4632c5c-b04c-4fea-af34-adb309454fbc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:01.250127 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4632c5c-b04c-4fea-af34-adb309454fbc admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:01.250430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4632c5c-b04c-4fea-af34-adb309454fbc admin admin] PATH_INFO: `/v3/users/b74c5a362206457ca5475eab90e3fda5` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:01.251611 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4632c5c-b04c-4fea-af34-adb309454fbc admin admin] RBAC: Authorizing `identity:delete_user(user_id=b74c5a362206457ca5475eab90e3fda5)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:16:01.253518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4632c5c-b04c-4fea-af34-adb309454fbc admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:16:01.301731 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e4632c5c-b04c-4fea-af34-adb309454fbc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b74c5a362206457ca5475eab90e3fda5'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:01.313994 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e4632c5c-b04c-4fea-af34-adb309454fbc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b74c5a362206457ca5475eab90e3fda5'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:01.320783 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e4632c5c-b04c-4fea-af34-adb309454fbc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b74c5a362206457ca5475eab90e3fda5'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:01.322007 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e4632c5c-b04c-4fea-af34-adb309454fbc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b74c5a362206457ca5475eab90e3fda5'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:01.328980 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e4632c5c-b04c-4fea-af34-adb309454fbc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b74c5a362206457ca5475eab90e3fda5'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:01.335003 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1258/2502] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:16:01 2022] DELETE /identity/v3/users/b74c5a362206457ca5475eab90e3fda5 => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:16:01.339637 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d97279-c94d-4f23-8c36-02546980a9dd None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:01.340159 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d97279-c94d-4f23-8c36-02546980a9dd None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:01.340265 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1d97279-c94d-4f23-8c36-02546980a9dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:01.357488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-c1d97279-c94d-4f23-8c36-02546980a9dd None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:16:01.358748 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c1d97279-c94d-4f23-8c36-02546980a9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:01.423634 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1245/2503] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:16:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:16:01.433972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4632c5c-b04c-4fea-af34-adb309454fbc admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:16:01.435412 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e4632c5c-b04c-4fea-af34-adb309454fbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:01.491945 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4632c5c-b04c-4fea-af34-adb309454fbc admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:16:01.496242 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b5561a-e36c-4f4b-9e8a-2db62262aba9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:16:01.496827 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5561a-e36c-4f4b-9e8a-2db62262aba9 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:01.497109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5561a-e36c-4f4b-9e8a-2db62262aba9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:01.497379 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b5561a-e36c-4f4b-9e8a-2db62262aba9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:01.504260 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:16:01.506837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b5561a-e36c-4f4b-9e8a-2db62262aba9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:16:01.508776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51b5561a-e36c-4f4b-9e8a-2db62262aba9 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:16:01.520638 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1259/2504] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:16:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:16:01.796527 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d97279-c94d-4f23-8c36-02546980a9dd None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:16:01.798129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c1d97279-c94d-4f23-8c36-02546980a9dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:01.811070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1d97279-c94d-4f23-8c36-02546980a9dd None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:16:01.816314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f8899-5c35-4f05-abf5-f61857b933f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:16:01.817532 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f8899-5c35-4f05-abf5-f61857b933f5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:01.817970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f8899-5c35-4f05-abf5-f61857b933f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:01.818247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d24f8899-5c35-4f05-abf5-f61857b933f5 admin admin] PATH_INFO: `/v3/projects/db8a8aee07ce44359359190b21aa5f82` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:01.820168 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f8899-5c35-4f05-abf5-f61857b933f5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=db8a8aee07ce44359359190b21aa5f82)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:16:01.822395 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d24f8899-5c35-4f05-abf5-f61857b933f5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:16:01.856312 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:16:01.883322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d24f8899-5c35-4f05-abf5-f61857b933f5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'db8a8aee07ce44359359190b21aa5f82', 'request_id': 'req-d24f8899-5c35-4f05-abf5-f61857b933f5'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:01.889352 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d24f8899-5c35-4f05-abf5-f61857b933f5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'db8a8aee07ce44359359190b21aa5f82', 'request_id': 'req-d24f8899-5c35-4f05-abf5-f61857b933f5'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:01.898756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d24f8899-5c35-4f05-abf5-f61857b933f5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'db8a8aee07ce44359359190b21aa5f82', 'request_id': 'req-d24f8899-5c35-4f05-abf5-f61857b933f5'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:01.915508 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d24f8899-5c35-4f05-abf5-f61857b933f5 admin admin] The token cache is being invalidate because project db8a8aee07ce44359359190b21aa5f82 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:16:01.915803 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d24f8899-5c35-4f05-abf5-f61857b933f5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:01.916464 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d24f8899-5c35-4f05-abf5-f61857b933f5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:01.918005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1246/2505] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:16:01 2022] DELETE /identity/v3/projects/db8a8aee07ce44359359190b21aa5f82 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:16:01.922026 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b5561a-e36c-4f4b-9e8a-2db62262aba9 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:16:01.932205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51b5561a-e36c-4f4b-9e8a-2db62262aba9 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:16:01.990566 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584af6f-5ef4-4ab2-a61d-0a3f95ccc85e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:16:01.991453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584af6f-5ef4-4ab2-a61d-0a3f95ccc85e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:01.991738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584af6f-5ef4-4ab2-a61d-0a3f95ccc85e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:01.992022 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f584af6f-5ef4-4ab2-a61d-0a3f95ccc85e admin admin] PATH_INFO: `/v3/users/7503ade196d942acaef4c43e08da58c2` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:01.993413 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584af6f-5ef4-4ab2-a61d-0a3f95ccc85e admin admin] RBAC: Authorizing `identity:delete_user(user_id=7503ade196d942acaef4c43e08da58c2)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:16:01.995551 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f584af6f-5ef4-4ab2-a61d-0a3f95ccc85e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:16:02.045076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f584af6f-5ef4-4ab2-a61d-0a3f95ccc85e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7503ade196d942acaef4c43e08da58c2'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:02.058960 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f584af6f-5ef4-4ab2-a61d-0a3f95ccc85e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7503ade196d942acaef4c43e08da58c2'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:02.067812 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f584af6f-5ef4-4ab2-a61d-0a3f95ccc85e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7503ade196d942acaef4c43e08da58c2'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:02.068652 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f584af6f-5ef4-4ab2-a61d-0a3f95ccc85e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7503ade196d942acaef4c43e08da58c2'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:02.077291 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f584af6f-5ef4-4ab2-a61d-0a3f95ccc85e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7503ade196d942acaef4c43e08da58c2'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:02.084411 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1260/2506] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:16:01 2022] DELETE /identity/v3/users/7503ade196d942acaef4c43e08da58c2 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:16:02.345417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f8899-5c35-4f05-abf5-f61857b933f5 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:16:02.346974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d24f8899-5c35-4f05-abf5-f61857b933f5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:02.360895 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d24f8899-5c35-4f05-abf5-f61857b933f5 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:16:02.419370 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f743b40-5ee6-4e93-aa4c-787e991cfc0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:16:02.420231 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f743b40-5ee6-4e93-aa4c-787e991cfc0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:02.420515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f743b40-5ee6-4e93-aa4c-787e991cfc0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:02.420797 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f743b40-5ee6-4e93-aa4c-787e991cfc0d admin admin] PATH_INFO: `/v3/projects/c245dbdd7a9741ca8d0f72524c37113e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:02.421964 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f743b40-5ee6-4e93-aa4c-787e991cfc0d admin admin] RBAC: Authorizing `identity:delete_project(project_id=c245dbdd7a9741ca8d0f72524c37113e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:16:02.423954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f743b40-5ee6-4e93-aa4c-787e991cfc0d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:16:02.468936 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1f743b40-5ee6-4e93-aa4c-787e991cfc0d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c245dbdd7a9741ca8d0f72524c37113e', 'request_id': 'req-1f743b40-5ee6-4e93-aa4c-787e991cfc0d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:02.474337 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1f743b40-5ee6-4e93-aa4c-787e991cfc0d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c245dbdd7a9741ca8d0f72524c37113e', 'request_id': 'req-1f743b40-5ee6-4e93-aa4c-787e991cfc0d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:02.481743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1f743b40-5ee6-4e93-aa4c-787e991cfc0d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c245dbdd7a9741ca8d0f72524c37113e', 'request_id': 'req-1f743b40-5ee6-4e93-aa4c-787e991cfc0d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:02.492578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1f743b40-5ee6-4e93-aa4c-787e991cfc0d admin admin] The token cache is being invalidate because project c245dbdd7a9741ca8d0f72524c37113e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:16:02.492862 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1f743b40-5ee6-4e93-aa4c-787e991cfc0d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:02.494017 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-1f743b40-5ee6-4e93-aa4c-787e991cfc0d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:02.495546 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1247/2507] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:16:02 2022] DELETE /identity/v3/projects/c245dbdd7a9741ca8d0f72524c37113e => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:16:02.499531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584af6f-5ef4-4ab2-a61d-0a3f95ccc85e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:16:02.501094 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f584af6f-5ef4-4ab2-a61d-0a3f95ccc85e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:02.513918 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f584af6f-5ef4-4ab2-a61d-0a3f95ccc85e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:16:02.570256 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e65f09b-62f6-4cb8-a0ed-7b0fdb4da9c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:16:02.571090 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65f09b-62f6-4cb8-a0ed-7b0fdb4da9c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:02.571377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65f09b-62f6-4cb8-a0ed-7b0fdb4da9c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:02.571659 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e65f09b-62f6-4cb8-a0ed-7b0fdb4da9c1 admin admin] PATH_INFO: `/v3/users/5a85b1c3088e40f4b3a9bf3d63346a06` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:02.572850 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65f09b-62f6-4cb8-a0ed-7b0fdb4da9c1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a85b1c3088e40f4b3a9bf3d63346a06)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:16:02.574812 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e65f09b-62f6-4cb8-a0ed-7b0fdb4da9c1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:16:02.590180 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:16:02.620343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7e65f09b-62f6-4cb8-a0ed-7b0fdb4da9c1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a85b1c3088e40f4b3a9bf3d63346a06'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:02.632257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7e65f09b-62f6-4cb8-a0ed-7b0fdb4da9c1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a85b1c3088e40f4b3a9bf3d63346a06'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:02.639581 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7e65f09b-62f6-4cb8-a0ed-7b0fdb4da9c1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a85b1c3088e40f4b3a9bf3d63346a06'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:02.640180 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7e65f09b-62f6-4cb8-a0ed-7b0fdb4da9c1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a85b1c3088e40f4b3a9bf3d63346a06'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:02.647190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-7e65f09b-62f6-4cb8-a0ed-7b0fdb4da9c1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a85b1c3088e40f4b3a9bf3d63346a06'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:02.653064 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1261/2508] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:16:02 2022] DELETE /identity/v3/users/5a85b1c3088e40f4b3a9bf3d63346a06 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:16:02.931598 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f743b40-5ee6-4e93-aa4c-787e991cfc0d admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:16:02.933009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1f743b40-5ee6-4e93-aa4c-787e991cfc0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:02.947982 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f743b40-5ee6-4e93-aa4c-787e991cfc0d admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:16:02.992319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:16:03.012548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570d2fe-37b1-447a-894b-a115fefc52e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:16:03.013456 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570d2fe-37b1-447a-894b-a115fefc52e9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:03.013744 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570d2fe-37b1-447a-894b-a115fefc52e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:03.014018 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4570d2fe-37b1-447a-894b-a115fefc52e9 admin admin] PATH_INFO: `/v3/projects/1df755d98ad94dd2a0942b8fa8f01923` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:03.015281 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570d2fe-37b1-447a-894b-a115fefc52e9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1df755d98ad94dd2a0942b8fa8f01923)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:16:03.017216 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4570d2fe-37b1-447a-894b-a115fefc52e9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:16:03.065906 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4570d2fe-37b1-447a-894b-a115fefc52e9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1df755d98ad94dd2a0942b8fa8f01923', 'request_id': 'req-4570d2fe-37b1-447a-894b-a115fefc52e9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:03.071561 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4570d2fe-37b1-447a-894b-a115fefc52e9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1df755d98ad94dd2a0942b8fa8f01923', 'request_id': 'req-4570d2fe-37b1-447a-894b-a115fefc52e9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:03.083429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4570d2fe-37b1-447a-894b-a115fefc52e9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1df755d98ad94dd2a0942b8fa8f01923', 'request_id': 'req-4570d2fe-37b1-447a-894b-a115fefc52e9'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:03.107310 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4570d2fe-37b1-447a-894b-a115fefc52e9 admin admin] The token cache is being invalidate because project 1df755d98ad94dd2a0942b8fa8f01923 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:16:03.107556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4570d2fe-37b1-447a-894b-a115fefc52e9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:03.108189 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-4570d2fe-37b1-447a-894b-a115fefc52e9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:16:03.110026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1248/2509] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:16:02 2022] DELETE /identity/v3/projects/1df755d98ad94dd2a0942b8fa8f01923 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:16:06.934545 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ac30c-a3ad-484e-9297-211f5d75110b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:06.935015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ac30c-a3ad-484e-9297-211f5d75110b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:06.935129 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05ac30c-a3ad-484e-9297-211f5d75110b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:06.963661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-f05ac30c-a3ad-484e-9297-211f5d75110b None None] MFA Rules not processed for user `52076f59f68f4d77a0dadbf7a98d2b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:16:06.964916 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f05ac30c-a3ad-484e-9297-211f5d75110b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:07.021178 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1262/2510] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 11 14:16:06 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:16:07.068423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570d2fe-37b1-447a-894b-a115fefc52e9 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:16:07.069811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4570d2fe-37b1-447a-894b-a115fefc52e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:07.141246 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4570d2fe-37b1-447a-894b-a115fefc52e9 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:16:07.146480 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1f5b6-cb77-4e1c-b22f-190f0bb3ebf8 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c1369033a44c1c81d4c1c9ff2bbedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:16:07.147138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1f5b6-cb77-4e1c-b22f-190f0bb3ebf8 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:07.147428 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1f5b6-cb77-4e1c-b22f-190f0bb3ebf8 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:07.147711 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1f5b6-cb77-4e1c-b22f-190f0bb3ebf8 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:07.156296 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d1f5b6-cb77-4e1c-b22f-190f0bb3ebf8 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:16:07.157952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d1f5b6-cb77-4e1c-b22f-190f0bb3ebf8 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:16:07.167380 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1249/2511] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 11 14:16:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 14:16:09.522922 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa111554-750a-47de-87f4-4364f6d65629 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:09.523489 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa111554-750a-47de-87f4-4364f6d65629 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:09.523489 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa111554-750a-47de-87f4-4364f6d65629 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:09.543059 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-aa111554-750a-47de-87f4-4364f6d65629 None None] MFA Rules not processed for user `3c85d19cf42f4688bbc89a439d96ca69`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:16:09.544266 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-aa111554-750a-47de-87f4-4364f6d65629 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:09.610610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1263/2512] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:16:09 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:16:17.163258 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=64,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:16:19.600046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=9,keystone:SELECT=74,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:16:24.665579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4318-ced8-4adf-9b81-9ad9b21b5c91 None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:24.666187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4318-ced8-4adf-9b81-9ad9b21b5c91 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:24.666187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-186e4318-ced8-4adf-9b81-9ad9b21b5c91 None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:24.667522 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1250/2513] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 11 14:16:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:16:24.673859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8d9ff-889f-42bc-9881-9842142b7026 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:24.674397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8d9ff-889f-42bc-9881-9842142b7026 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:24.674510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fff8d9ff-889f-42bc-9881-9842142b7026 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:24.693935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-fff8d9ff-889f-42bc-9881-9842142b7026 None None] MFA Rules not processed for user `ef5dac3f311e43809a46995bab56e780`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:16:24.695328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-fff8d9ff-889f-42bc-9881-9842142b7026 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:24.887162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1264/2514] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 11 14:16:24 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:16:24.898508 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e4318-ced8-4adf-9b81-9ad9b21b5c91 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:16:24.900816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-186e4318-ced8-4adf-9b81-9ad9b21b5c91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:24.959219 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-186e4318-ced8-4adf-9b81-9ad9b21b5c91 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:16:24.963596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81df2ccd-59fa-4600-aa10-a37de7916d42 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f80d59fec4994e689e857ba2087d3a23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:16:24.964242 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df2ccd-59fa-4600-aa10-a37de7916d42 service glance] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:24.964532 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df2ccd-59fa-4600-aa10-a37de7916d42 service glance] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:24.964811 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81df2ccd-59fa-4600-aa10-a37de7916d42 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:24.975104 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df2ccd-59fa-4600-aa10-a37de7916d42 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:16:24.976610 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81df2ccd-59fa-4600-aa10-a37de7916d42 service glance] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:16:24.987751 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1251/2515] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:16:24 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:16:25.053228 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dce39-13a9-4580-acc4-8d93baed1ef5 None None] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:25.053539 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dce39-13a9-4580-acc4-8d93baed1ef5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:25.053819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4dce39-13a9-4580-acc4-8d93baed1ef5 None None] PATH_INFO: `/` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:25.055319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1265/2516] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 11 14:16:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:16:25.060096 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc734364-e0b2-4f62-b0a8-9aa8f3e63840 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:25.060382 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc734364-e0b2-4f62-b0a8-9aa8f3e63840 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:25.060655 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc734364-e0b2-4f62-b0a8-9aa8f3e63840 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:25.080089 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-bc734364-e0b2-4f62-b0a8-9aa8f3e63840 None None] MFA Rules not processed for user `ce1c85ee44ef45d2b631e51112b5ea0a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:16:25.081310 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bc734364-e0b2-4f62-b0a8-9aa8f3e63840 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:25.134441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1252/2517] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 11 14:16:25 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:16:25.148244 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dce39-13a9-4580-acc4-8d93baed1ef5 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:16:25.150776 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9f4dce39-13a9-4580-acc4-8d93baed1ef5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:25.168222 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4dce39-13a9-4580-acc4-8d93baed1ef5 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:16:25.172413 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17c86290-064a-464e-ad13-b0e1014b421f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:16:25.173173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c86290-064a-464e-ad13-b0e1014b421f service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:25.173493 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c86290-064a-464e-ad13-b0e1014b421f service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:25.173841 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17c86290-064a-464e-ad13-b0e1014b421f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:25.181879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c86290-064a-464e-ad13-b0e1014b421f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:16:25.183383 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17c86290-064a-464e-ad13-b0e1014b421f service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:16:25.191624 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1266/2518] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:16:25 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:16:25.279954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53467248-f211-477a-a73d-a0cc6d24123c None None] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:25.280770 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53467248-f211-477a-a73d-a0cc6d24123c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:25.281498 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53467248-f211-477a-a73d-a0cc6d24123c None None] PATH_INFO: `/` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:25.285066 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1253/2519] 10.222.0.49 () {54 vars in 921 bytes} [Mon Jul 11 14:16:25 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 11 14:16:25.293134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1500908-4c26-4918-8da6-a3a7500b88dc None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:25.293460 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1500908-4c26-4918-8da6-a3a7500b88dc None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:25.293703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1500908-4c26-4918-8da6-a3a7500b88dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:25.312382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b1500908-4c26-4918-8da6-a3a7500b88dc None None] MFA Rules not processed for user `ef5dac3f311e43809a46995bab56e780`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:16:25.313598 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b1500908-4c26-4918-8da6-a3a7500b88dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:25.323167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1267/2520] 10.222.0.49 () {58 vars in 1018 bytes} [Mon Jul 11 14:16:25 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:16:25.334088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53467248-f211-477a-a73d-a0cc6d24123c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:16:25.336136 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-53467248-f211-477a-a73d-a0cc6d24123c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:25.350215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53467248-f211-477a-a73d-a0cc6d24123c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:16:25.354450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa742a25-5f33-490b-ac7b-a19b52c8c15d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:16:25.355083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa742a25-5f33-490b-ac7b-a19b52c8c15d service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:25.355378 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa742a25-5f33-490b-ac7b-a19b52c8c15d service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:25.355657 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa742a25-5f33-490b-ac7b-a19b52c8c15d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:25.365851 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa742a25-5f33-490b-ac7b-a19b52c8c15d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:16:25.367303 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa742a25-5f33-490b-ac7b-a19b52c8c15d service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:16:25.378005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1254/2521] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:16:25 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:16:27.139804 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999f8fa-ae49-4908-bdda-14675f5cb5fa None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:27.140408 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999f8fa-ae49-4908-bdda-14675f5cb5fa None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:27.140408 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0999f8fa-ae49-4908-bdda-14675f5cb5fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:27.161511 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-0999f8fa-ae49-4908-bdda-14675f5cb5fa None None] MFA Rules not processed for user `52076f59f68f4d77a0dadbf7a98d2b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:16:27.162769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0999f8fa-ae49-4908-bdda-14675f5cb5fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:27.173394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1268/2522] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 11 14:16:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:16:27.182834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa742a25-5f33-490b-ac7b-a19b52c8c15d service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:16:27.194655 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa742a25-5f33-490b-ac7b-a19b52c8c15d service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:16:27.200237 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5ba506-1cdf-48f9-9009-40b3ea517fa4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c1369033a44c1c81d4c1c9ff2bbedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:16:27.200898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ba506-1cdf-48f9-9009-40b3ea517fa4 service swift] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:27.201184 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ba506-1cdf-48f9-9009-40b3ea517fa4 service swift] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:27.201459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5ba506-1cdf-48f9-9009-40b3ea517fa4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:27.210616 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ba506-1cdf-48f9-9009-40b3ea517fa4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:16:27.212841 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c5ba506-1cdf-48f9-9009-40b3ea517fa4 service swift] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:16:27.221911 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1255/2523] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 11 14:16:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 14:16:37.155927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=43 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:16:37.218216 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=61 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:16:46.047215 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999f8fa-ae49-4908-bdda-14675f5cb5fa None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:16:46.058631 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0999f8fa-ae49-4908-bdda-14675f5cb5fa None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:16:46.062387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54187ed4-618a-4c3c-b24b-9d3a076403e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:16:46.063064 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54187ed4-618a-4c3c-b24b-9d3a076403e7 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:46.063346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54187ed4-618a-4c3c-b24b-9d3a076403e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:46.063618 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54187ed4-618a-4c3c-b24b-9d3a076403e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:46.065378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-54187ed4-618a-4c3c-b24b-9d3a076403e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:46.154528 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54187ed4-618a-4c3c-b24b-9d3a076403e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:16:46.155985 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54187ed4-618a-4c3c-b24b-9d3a076403e7 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:16:46.169364 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1269/2524] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:16:46 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:16:47.602131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca85543-9c98-40a9-8b9b-0504fd55f1a7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:47.603026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca85543-9c98-40a9-8b9b-0504fd55f1a7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:47.603026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eca85543-9c98-40a9-8b9b-0504fd55f1a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:47.622103 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-eca85543-9c98-40a9-8b9b-0504fd55f1a7 None None] MFA Rules not processed for user `52076f59f68f4d77a0dadbf7a98d2b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:16:47.623409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-eca85543-9c98-40a9-8b9b-0504fd55f1a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:47.631211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1256/2525] 10.222.0.49 () {58 vars in 967 bytes} [Mon Jul 11 14:16:47 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:16:47.645532 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54187ed4-618a-4c3c-b24b-9d3a076403e7 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:16:47.661871 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54187ed4-618a-4c3c-b24b-9d3a076403e7 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:16:47.667735 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d882b9c-6b81-4c9c-807f-658309076c72 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c1369033a44c1c81d4c1c9ff2bbedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:16:47.668399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d882b9c-6b81-4c9c-807f-658309076c72 service swift] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:47.668684 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d882b9c-6b81-4c9c-807f-658309076c72 service swift] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:47.668963 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d882b9c-6b81-4c9c-807f-658309076c72 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:47.678053 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d882b9c-6b81-4c9c-807f-658309076c72 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:16:47.679686 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d882b9c-6b81-4c9c-807f-658309076c72 service swift] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:16:47.688038 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1270/2526] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 11 14:16:47 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 14:16:51.838186 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca85543-9c98-40a9-8b9b-0504fd55f1a7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:16:51.848092 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eca85543-9c98-40a9-8b9b-0504fd55f1a7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:16:51.853558 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01dd3b7-e887-4f25-b830-145fc3f9fb98 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:16:51.854570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01dd3b7-e887-4f25-b830-145fc3f9fb98 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:51.855017 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01dd3b7-e887-4f25-b830-145fc3f9fb98 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:51.855529 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01dd3b7-e887-4f25-b830-145fc3f9fb98 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:51.858090 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f01dd3b7-e887-4f25-b830-145fc3f9fb98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:51.876265 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01dd3b7-e887-4f25-b830-145fc3f9fb98 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:16:51.877759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01dd3b7-e887-4f25-b830-145fc3f9fb98 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:16:51.889859 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1257/2527] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:16:51 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:16:57.684697 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=36 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:16:58.379420 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d882b9c-6b81-4c9c-807f-658309076c72 service swift] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:16:58.388693 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d882b9c-6b81-4c9c-807f-658309076c72 service swift] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:16:58.392200 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78028745-469a-4f6f-94f4-f54a6de42932 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:16:58.392831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78028745-469a-4f6f-94f4-f54a6de42932 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:16:58.393115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78028745-469a-4f6f-94f4-f54a6de42932 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:16:58.393393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78028745-469a-4f6f-94f4-f54a6de42932 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:16:58.395181 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-78028745-469a-4f6f-94f4-f54a6de42932 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:16:58.457941 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78028745-469a-4f6f-94f4-f54a6de42932 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:16:58.459436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78028745-469a-4f6f-94f4-f54a6de42932 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:16:58.469180 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1271/2528] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:16:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:01.657906 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01dd3b7-e887-4f25-b830-145fc3f9fb98 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:01.662343 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f01dd3b7-e887-4f25-b830-145fc3f9fb98 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:01.679227 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01dd3b7-e887-4f25-b830-145fc3f9fb98 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:01.684440 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09573f9-2920-4a8c-bac8-6a617e64d34d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:01.685327 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09573f9-2920-4a8c-bac8-6a617e64d34d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:01.685670 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09573f9-2920-4a8c-bac8-6a617e64d34d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:01.685954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09573f9-2920-4a8c-bac8-6a617e64d34d admin admin] PATH_INFO: `/v3/users/296853a88430438e96e0f8fb97b46d2e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:01.703611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09573f9-2920-4a8c-bac8-6a617e64d34d admin admin] RBAC: Authorizing `identity:delete_user(user_id=296853a88430438e96e0f8fb97b46d2e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:01.705588 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09573f9-2920-4a8c-bac8-6a617e64d34d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:01.757875 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a09573f9-2920-4a8c-bac8-6a617e64d34d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '296853a88430438e96e0f8fb97b46d2e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:01.759340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a09573f9-2920-4a8c-bac8-6a617e64d34d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '296853a88430438e96e0f8fb97b46d2e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:01.767426 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a09573f9-2920-4a8c-bac8-6a617e64d34d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '296853a88430438e96e0f8fb97b46d2e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:01.768082 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a09573f9-2920-4a8c-bac8-6a617e64d34d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '296853a88430438e96e0f8fb97b46d2e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:01.780685 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a09573f9-2920-4a8c-bac8-6a617e64d34d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '296853a88430438e96e0f8fb97b46d2e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:01.792904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1258/2529] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:17:01 2022] DELETE /identity/v3/users/296853a88430438e96e0f8fb97b46d2e => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:01.975541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78028745-469a-4f6f-94f4-f54a6de42932 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:01.987451 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78028745-469a-4f6f-94f4-f54a6de42932 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:02.044392 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89979b5f-932b-4818-ba9d-686e81837214 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:02.045180 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89979b5f-932b-4818-ba9d-686e81837214 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:02.045472 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89979b5f-932b-4818-ba9d-686e81837214 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:02.045756 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89979b5f-932b-4818-ba9d-686e81837214 admin admin] PATH_INFO: `/v3/projects/a772cbaa461c416da2cc35e364541ecf` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:02.057430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89979b5f-932b-4818-ba9d-686e81837214 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a772cbaa461c416da2cc35e364541ecf)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:02.059419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89979b5f-932b-4818-ba9d-686e81837214 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:02.108703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-89979b5f-932b-4818-ba9d-686e81837214 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a772cbaa461c416da2cc35e364541ecf', 'request_id': 'req-89979b5f-932b-4818-ba9d-686e81837214'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:02.115929 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-89979b5f-932b-4818-ba9d-686e81837214 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a772cbaa461c416da2cc35e364541ecf', 'request_id': 'req-89979b5f-932b-4818-ba9d-686e81837214'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:02.121913 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-89979b5f-932b-4818-ba9d-686e81837214 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a772cbaa461c416da2cc35e364541ecf', 'request_id': 'req-89979b5f-932b-4818-ba9d-686e81837214'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:02.133273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-89979b5f-932b-4818-ba9d-686e81837214 admin admin] The token cache is being invalidate because project a772cbaa461c416da2cc35e364541ecf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:17:02.133558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-89979b5f-932b-4818-ba9d-686e81837214 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:02.134196 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-89979b5f-932b-4818-ba9d-686e81837214 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:02.135746 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1272/2530] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:17:01 2022] DELETE /identity/v3/projects/a772cbaa461c416da2cc35e364541ecf => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:02.140699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4062f-20d9-47bc-997b-010aa2cafd27 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:02.140994 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4062f-20d9-47bc-997b-010aa2cafd27 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:02.141276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b4062f-20d9-47bc-997b-010aa2cafd27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:02.160898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-79b4062f-20d9-47bc-997b-010aa2cafd27 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:02.162174 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-79b4062f-20d9-47bc-997b-010aa2cafd27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:02.220963 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1259/2531] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:02.230620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229aa9a-e88e-482a-a2a3-ced8e0409ab8 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:02.231073 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229aa9a-e88e-482a-a2a3-ced8e0409ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:02.231385 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c229aa9a-e88e-482a-a2a3-ced8e0409ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:02.249953 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c229aa9a-e88e-482a-a2a3-ced8e0409ab8 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:02.251203 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c229aa9a-e88e-482a-a2a3-ced8e0409ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:02.259175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1273/2532] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:02.263545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4062f-20d9-47bc-997b-010aa2cafd27 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:02.273409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b4062f-20d9-47bc-997b-010aa2cafd27 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:02.278009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70d082-ddad-475d-b810-7cad4739eec1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:02.278699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70d082-ddad-475d-b810-7cad4739eec1 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:02.278994 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70d082-ddad-475d-b810-7cad4739eec1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:02.279272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be70d082-ddad-475d-b810-7cad4739eec1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:02.280001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70d082-ddad-475d-b810-7cad4739eec1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:17:02.280310 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70d082-ddad-475d-b810-7cad4739eec1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:02.281898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be70d082-ddad-475d-b810-7cad4739eec1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:02.294118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1260/2533] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:17:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 14:17:02.298569 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229aa9a-e88e-482a-a2a3-ced8e0409ab8 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:02.308669 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c229aa9a-e88e-482a-a2a3-ced8e0409ab8 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:02.313584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac62b6-1e7b-4a2a-82f0-e650d3dc358a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:02.314235 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac62b6-1e7b-4a2a-82f0-e650d3dc358a admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:02.314538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac62b6-1e7b-4a2a-82f0-e650d3dc358a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:02.314833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ac62b6-1e7b-4a2a-82f0-e650d3dc358a admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:02.315531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac62b6-1e7b-4a2a-82f0-e650d3dc358a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:02.317414 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ac62b6-1e7b-4a2a-82f0-e650d3dc358a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:02.321354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:02.353360 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1274/2534] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:17:02 2022] POST /identity/v3/projects => generated 405 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:17:02.357641 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70d082-ddad-475d-b810-7cad4739eec1 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:02.368769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be70d082-ddad-475d-b810-7cad4739eec1 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:02.395726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:02.428662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60c1fe-7d7e-4b3e-b5a7-2d12a752fb51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:02.429969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60c1fe-7d7e-4b3e-b5a7-2d12a752fb51 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:02.430259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60c1fe-7d7e-4b3e-b5a7-2d12a752fb51 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:02.430571 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf60c1fe-7d7e-4b3e-b5a7-2d12a752fb51 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:02.431264 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60c1fe-7d7e-4b3e-b5a7-2d12a752fb51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:02.433173 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf60c1fe-7d7e-4b3e-b5a7-2d12a752fb51 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:02.453262 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1261/2535] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:17:02 2022] POST /identity/v3/users => generated 303 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:17:02.457376 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac62b6-1e7b-4a2a-82f0-e650d3dc358a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:02.468799 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ac62b6-1e7b-4a2a-82f0-e650d3dc358a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:02.473552 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40756f-d59a-4b15-9403-3ac8a6f376a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:02.474215 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40756f-d59a-4b15-9403-3ac8a6f376a5 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:02.474539 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40756f-d59a-4b15-9403-3ac8a6f376a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:02.474822 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e40756f-d59a-4b15-9403-3ac8a6f376a5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:02.476303 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40756f-d59a-4b15-9403-3ac8a6f376a5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:02.477903 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e40756f-d59a-4b15-9403-3ac8a6f376a5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:02.487963 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1275/2536] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:02.491993 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60c1fe-7d7e-4b3e-b5a7-2d12a752fb51 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:02.501939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf60c1fe-7d7e-4b3e-b5a7-2d12a752fb51 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:02.506666 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd78fd8-0e31-47f4-af5f-2865adf76823 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:02.507322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd78fd8-0e31-47f4-af5f-2865adf76823 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:02.507630 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd78fd8-0e31-47f4-af5f-2865adf76823 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:02.507917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd78fd8-0e31-47f4-af5f-2865adf76823 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:02.508651 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd78fd8-0e31-47f4-af5f-2865adf76823 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:02.510271 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd78fd8-0e31-47f4-af5f-2865adf76823 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:02.520693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1262/2537] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:02.525283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40756f-d59a-4b15-9403-3ac8a6f376a5 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:02.535310 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e40756f-d59a-4b15-9403-3ac8a6f376a5 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:02.540690 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06c5b8-d970-468c-9a70-9829f8a37ce8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:02.541581 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06c5b8-d970-468c-9a70-9829f8a37ce8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:02.541868 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06c5b8-d970-468c-9a70-9829f8a37ce8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:02.542147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa06c5b8-d970-468c-9a70-9829f8a37ce8 admin admin] PATH_INFO: `/v3/projects/7275ede0054849468e6e31d51718fb65/users/3cb6327cf37e4c3a888557209348cfa0/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:02.561593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa06c5b8-d970-468c-9a70-9829f8a37ce8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7275ede0054849468e6e31d51718fb65, user_id=3cb6327cf37e4c3a888557209348cfa0, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:02.564373 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa06c5b8-d970-468c-9a70-9829f8a37ce8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:02.579300 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1276/2538] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:02 2022] PUT /identity/v3/projects/7275ede0054849468e6e31d51718fb65/users/3cb6327cf37e4c3a888557209348cfa0/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:02.590009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412400c1-6cfd-4973-83b3-f23f07757893 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:02.590278 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412400c1-6cfd-4973-83b3-f23f07757893 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:02.590586 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-412400c1-6cfd-4973-83b3-f23f07757893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:02.617900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-412400c1-6cfd-4973-83b3-f23f07757893 None None] MFA Rules not processed for user `3cb6327cf37e4c3a888557209348cfa0`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:02.619165 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-412400c1-6cfd-4973-83b3-f23f07757893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:02.685890 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1263/2539] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:02 2022] POST /identity/v3/auth/tokens => generated 3272 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:02.690415 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06c5b8-d970-468c-9a70-9829f8a37ce8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:02.751615 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa06c5b8-d970-468c-9a70-9829f8a37ce8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:02.756281 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e5cfa-aefe-4cbc-b986-84814ab83737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:02.756979 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e5cfa-aefe-4cbc-b986-84814ab83737 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:02.757261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e5cfa-aefe-4cbc-b986-84814ab83737 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:02.757539 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51e5cfa-aefe-4cbc-b986-84814ab83737 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:02.758237 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e5cfa-aefe-4cbc-b986-84814ab83737 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:02.760196 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51e5cfa-aefe-4cbc-b986-84814ab83737 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:02.784939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1277/2540] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:17:02 2022] POST /identity/v3/projects => generated 405 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:02.789590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412400c1-6cfd-4973-83b3-f23f07757893 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:02.800605 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-412400c1-6cfd-4973-83b3-f23f07757893 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:02.856113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabb08c-267d-4529-87ed-3c3575c1adf3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:02.856748 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabb08c-267d-4529-87ed-3c3575c1adf3 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:02.857037 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabb08c-267d-4529-87ed-3c3575c1adf3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:02.857322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dabb08c-267d-4529-87ed-3c3575c1adf3 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:02.858502 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dabb08c-267d-4529-87ed-3c3575c1adf3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:02.860492 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dabb08c-267d-4529-87ed-3c3575c1adf3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:02.882800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1264/2541] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:17:02 2022] POST /identity/v3/users => generated 309 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:02.886841 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e5cfa-aefe-4cbc-b986-84814ab83737 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:02.899946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51e5cfa-aefe-4cbc-b986-84814ab83737 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:02.904671 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e1df2-21ca-44d6-8aba-e03943e38c61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:02.905338 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e1df2-21ca-44d6-8aba-e03943e38c61 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:02.905626 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e1df2-21ca-44d6-8aba-e03943e38c61 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:02.905935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40e1df2-21ca-44d6-8aba-e03943e38c61 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:02.906700 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e1df2-21ca-44d6-8aba-e03943e38c61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:02.908311 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40e1df2-21ca-44d6-8aba-e03943e38c61 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:02.919231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1278/2542] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:02.923390 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabb08c-267d-4529-87ed-3c3575c1adf3 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:02.933595 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dabb08c-267d-4529-87ed-3c3575c1adf3 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:02.938205 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f215a95-e384-40c1-8fae-781318d44524 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:02.939111 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f215a95-e384-40c1-8fae-781318d44524 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:02.939396 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f215a95-e384-40c1-8fae-781318d44524 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:02.939677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f215a95-e384-40c1-8fae-781318d44524 admin admin] PATH_INFO: `/v3/projects/49a17dfb538f4e3d8547f1140a1bb3cc/users/0fb2241d39194f0e94879d57e58a5d61/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:02.955169 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f215a95-e384-40c1-8fae-781318d44524 admin admin] RBAC: Authorizing `identity:create_grant(project_id=49a17dfb538f4e3d8547f1140a1bb3cc, user_id=0fb2241d39194f0e94879d57e58a5d61, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:02.957795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f215a95-e384-40c1-8fae-781318d44524 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:02.971688 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1265/2543] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:02 2022] PUT /identity/v3/projects/49a17dfb538f4e3d8547f1140a1bb3cc/users/0fb2241d39194f0e94879d57e58a5d61/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:02.975591 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e1df2-21ca-44d6-8aba-e03943e38c61 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:02.985989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40e1df2-21ca-44d6-8aba-e03943e38c61 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:03.050965 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bd1ea94-7e28-4946-8359-040b8700a7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:03.051884 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1ea94-7e28-4946-8359-040b8700a7ff admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:03.052380 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1ea94-7e28-4946-8359-040b8700a7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:03.052660 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bd1ea94-7e28-4946-8359-040b8700a7ff admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:03.053407 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1ea94-7e28-4946-8359-040b8700a7ff admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:03.055148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bd1ea94-7e28-4946-8359-040b8700a7ff admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:03.065618 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1279/2544] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:02 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:03.070903 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f215a95-e384-40c1-8fae-781318d44524 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:03.076969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:03.084060 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f215a95-e384-40c1-8fae-781318d44524 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:03.089951 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5da0e6-6ed9-450a-8a15-69a04c0a414b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:03.090872 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5da0e6-6ed9-450a-8a15-69a04c0a414b admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:03.091160 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5da0e6-6ed9-450a-8a15-69a04c0a414b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:03.091436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f5da0e6-6ed9-450a-8a15-69a04c0a414b admin admin] PATH_INFO: `/v3/projects/49a17dfb538f4e3d8547f1140a1bb3cc/users/0fb2241d39194f0e94879d57e58a5d61/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:03.093429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5da0e6-6ed9-450a-8a15-69a04c0a414b admin admin] RBAC: Authorizing `identity:create_grant(project_id=49a17dfb538f4e3d8547f1140a1bb3cc, user_id=0fb2241d39194f0e94879d57e58a5d61, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:03.096091 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f5da0e6-6ed9-450a-8a15-69a04c0a414b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:03.108428 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1266/2545] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:03 2022] PUT /identity/v3/projects/49a17dfb538f4e3d8547f1140a1bb3cc/users/0fb2241d39194f0e94879d57e58a5d61/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:03.120047 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ddbc2-2352-4e72-89a3-6c3e905ba5ff None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:03.120334 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ddbc2-2352-4e72-89a3-6c3e905ba5ff None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:03.120611 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9ddbc2-2352-4e72-89a3-6c3e905ba5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:03.145030 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:03.149854 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-aa9ddbc2-2352-4e72-89a3-6c3e905ba5ff None None] MFA Rules not processed for user `0fb2241d39194f0e94879d57e58a5d61`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:03.151590 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-aa9ddbc2-2352-4e72-89a3-6c3e905ba5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:03.220572 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1280/2546] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:03 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:03.231187 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5da0e6-6ed9-450a-8a15-69a04c0a414b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:03.232899 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5f5da0e6-6ed9-450a-8a15-69a04c0a414b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:03.290516 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f5da0e6-6ed9-450a-8a15-69a04c0a414b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:03.294174 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36050135-ebe3-4f98-be80-4081da68055d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:03.294854 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36050135-ebe3-4f98-be80-4081da68055d service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:03.295138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36050135-ebe3-4f98-be80-4081da68055d service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:03.295416 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36050135-ebe3-4f98-be80-4081da68055d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:03.304575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36050135-ebe3-4f98-be80-4081da68055d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:03.306515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36050135-ebe3-4f98-be80-4081da68055d service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:03.316058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1267/2547] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:03 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:03.468612 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ddbc2-2352-4e72-89a3-6c3e905ba5ff None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:03.528912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9ddbc2-2352-4e72-89a3-6c3e905ba5ff None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:03.533490 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e404e3-260b-4177-886f-a535718af0b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:03.534382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e404e3-260b-4177-886f-a535718af0b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:03.534675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e404e3-260b-4177-886f-a535718af0b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:03.534955 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e404e3-260b-4177-886f-a535718af0b4 admin admin] PATH_INFO: `/v3/users/3cb6327cf37e4c3a888557209348cfa0` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:03.536144 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e404e3-260b-4177-886f-a535718af0b4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3cb6327cf37e4c3a888557209348cfa0)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:03.538064 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e404e3-260b-4177-886f-a535718af0b4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:03.584052 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d8e404e3-260b-4177-886f-a535718af0b4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3cb6327cf37e4c3a888557209348cfa0'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:03.595648 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d8e404e3-260b-4177-886f-a535718af0b4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3cb6327cf37e4c3a888557209348cfa0'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:03.604392 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d8e404e3-260b-4177-886f-a535718af0b4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3cb6327cf37e4c3a888557209348cfa0'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:03.605004 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d8e404e3-260b-4177-886f-a535718af0b4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3cb6327cf37e4c3a888557209348cfa0'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:03.612165 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d8e404e3-260b-4177-886f-a535718af0b4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3cb6327cf37e4c3a888557209348cfa0'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:03.618600 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1281/2548] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:17:03 2022] DELETE /identity/v3/users/3cb6327cf37e4c3a888557209348cfa0 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:03.623122 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a7379-c603-46e4-bc14-a376bbc6f5c1 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:03.623377 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a7379-c603-46e4-bc14-a376bbc6f5c1 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:03.623652 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd8a7379-c603-46e4-bc14-a376bbc6f5c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:03.640675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cd8a7379-c603-46e4-bc14-a376bbc6f5c1 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:03.641862 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cd8a7379-c603-46e4-bc14-a376bbc6f5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:03.699674 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1268/2549] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:03.709200 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e404e3-260b-4177-886f-a535718af0b4 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:03.710697 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d8e404e3-260b-4177-886f-a535718af0b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:03.731227 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:03.769789 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e404e3-260b-4177-886f-a535718af0b4 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:03.773457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90746a5e-50e6-44a3-bcd7-02a626a87ff3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:03.774160 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90746a5e-50e6-44a3-bcd7-02a626a87ff3 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:03.774512 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90746a5e-50e6-44a3-bcd7-02a626a87ff3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:03.774838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90746a5e-50e6-44a3-bcd7-02a626a87ff3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:03.784590 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90746a5e-50e6-44a3-bcd7-02a626a87ff3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:03.786028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90746a5e-50e6-44a3-bcd7-02a626a87ff3 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:03.795963 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1282/2550] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:04.064845 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a7379-c603-46e4-bc14-a376bbc6f5c1 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:04.066449 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cd8a7379-c603-46e4-bc14-a376bbc6f5c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:04.081502 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd8a7379-c603-46e4-bc14-a376bbc6f5c1 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:04.086341 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa91421-e979-4174-a1ce-0ab47db0c620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:04.087676 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa91421-e979-4174-a1ce-0ab47db0c620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:04.087676 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa91421-e979-4174-a1ce-0ab47db0c620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:04.088012 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaa91421-e979-4174-a1ce-0ab47db0c620 admin admin] PATH_INFO: `/v3/projects/7275ede0054849468e6e31d51718fb65` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:04.089379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa91421-e979-4174-a1ce-0ab47db0c620 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7275ede0054849468e6e31d51718fb65)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:04.091963 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaa91421-e979-4174-a1ce-0ab47db0c620 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:04.141887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eaa91421-e979-4174-a1ce-0ab47db0c620 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7275ede0054849468e6e31d51718fb65', 'request_id': 'req-eaa91421-e979-4174-a1ce-0ab47db0c620'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:04.147360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eaa91421-e979-4174-a1ce-0ab47db0c620 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7275ede0054849468e6e31d51718fb65', 'request_id': 'req-eaa91421-e979-4174-a1ce-0ab47db0c620'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:04.155670 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eaa91421-e979-4174-a1ce-0ab47db0c620 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7275ede0054849468e6e31d51718fb65', 'request_id': 'req-eaa91421-e979-4174-a1ce-0ab47db0c620'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:04.168287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eaa91421-e979-4174-a1ce-0ab47db0c620 admin admin] The token cache is being invalidate because project 7275ede0054849468e6e31d51718fb65 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:17:04.168575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eaa91421-e979-4174-a1ce-0ab47db0c620 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:04.169219 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-eaa91421-e979-4174-a1ce-0ab47db0c620 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:04.170801 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1269/2551] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:17:04 2022] DELETE /identity/v3/projects/7275ede0054849468e6e31d51718fb65 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:17:04.175166 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90746a5e-50e6-44a3-bcd7-02a626a87ff3 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:04.185173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90746a5e-50e6-44a3-bcd7-02a626a87ff3 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:04.239661 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8333c633-f5c4-4334-8d34-1343b908d39e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:04.240525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333c633-f5c4-4334-8d34-1343b908d39e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:04.240821 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333c633-f5c4-4334-8d34-1343b908d39e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:04.241077 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8333c633-f5c4-4334-8d34-1343b908d39e admin admin] PATH_INFO: `/v3/users/0fb2241d39194f0e94879d57e58a5d61` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:04.242811 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8333c633-f5c4-4334-8d34-1343b908d39e admin admin] RBAC: Authorizing `identity:delete_user(user_id=0fb2241d39194f0e94879d57e58a5d61)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:04.244722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8333c633-f5c4-4334-8d34-1343b908d39e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:04.294367 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8333c633-f5c4-4334-8d34-1343b908d39e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0fb2241d39194f0e94879d57e58a5d61'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:04.306712 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8333c633-f5c4-4334-8d34-1343b908d39e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0fb2241d39194f0e94879d57e58a5d61'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:04.314004 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8333c633-f5c4-4334-8d34-1343b908d39e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0fb2241d39194f0e94879d57e58a5d61'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:04.314755 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8333c633-f5c4-4334-8d34-1343b908d39e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0fb2241d39194f0e94879d57e58a5d61'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:04.322350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8333c633-f5c4-4334-8d34-1343b908d39e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0fb2241d39194f0e94879d57e58a5d61'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:04.328184 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1283/2552] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:17:04 2022] DELETE /identity/v3/users/0fb2241d39194f0e94879d57e58a5d61 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:04.624407 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa91421-e979-4174-a1ce-0ab47db0c620 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:04.625838 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-eaa91421-e979-4174-a1ce-0ab47db0c620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:04.639233 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaa91421-e979-4174-a1ce-0ab47db0c620 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:04.656431 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:04.708971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5a153c1-4cbe-4e51-a8cc-f7d1c3164876 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:04.709778 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a153c1-4cbe-4e51-a8cc-f7d1c3164876 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:04.710047 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a153c1-4cbe-4e51-a8cc-f7d1c3164876 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:04.710357 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a153c1-4cbe-4e51-a8cc-f7d1c3164876 admin admin] PATH_INFO: `/v3/projects/49a17dfb538f4e3d8547f1140a1bb3cc` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:04.712091 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a153c1-4cbe-4e51-a8cc-f7d1c3164876 admin admin] RBAC: Authorizing `identity:delete_project(project_id=49a17dfb538f4e3d8547f1140a1bb3cc)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:04.713997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5a153c1-4cbe-4e51-a8cc-f7d1c3164876 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:04.762204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a5a153c1-4cbe-4e51-a8cc-f7d1c3164876 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '49a17dfb538f4e3d8547f1140a1bb3cc', 'request_id': 'req-a5a153c1-4cbe-4e51-a8cc-f7d1c3164876'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:04.768143 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a5a153c1-4cbe-4e51-a8cc-f7d1c3164876 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '49a17dfb538f4e3d8547f1140a1bb3cc', 'request_id': 'req-a5a153c1-4cbe-4e51-a8cc-f7d1c3164876'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:04.775767 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a5a153c1-4cbe-4e51-a8cc-f7d1c3164876 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '49a17dfb538f4e3d8547f1140a1bb3cc', 'request_id': 'req-a5a153c1-4cbe-4e51-a8cc-f7d1c3164876'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:04.787221 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a5a153c1-4cbe-4e51-a8cc-f7d1c3164876 admin admin] The token cache is being invalidate because project 49a17dfb538f4e3d8547f1140a1bb3cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:17:04.787523 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a5a153c1-4cbe-4e51-a8cc-f7d1c3164876 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:04.788156 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a5a153c1-4cbe-4e51-a8cc-f7d1c3164876 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:04.790353 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1270/2553] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:17:04 2022] DELETE /identity/v3/projects/49a17dfb538f4e3d8547f1140a1bb3cc => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:17:04.795750 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d8962-3184-45a7-845a-a1efdf050c39 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:04.796052 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d8962-3184-45a7-845a-a1efdf050c39 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:04.796332 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32d8962-3184-45a7-845a-a1efdf050c39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:04.816364 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e32d8962-3184-45a7-845a-a1efdf050c39 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:04.817604 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e32d8962-3184-45a7-845a-a1efdf050c39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:04.854130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:04.888767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1284/2554] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:04.894869 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbdb74-636b-4785-b610-30b153cdf480 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:04.895162 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbdb74-636b-4785-b610-30b153cdf480 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:04.895444 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0fbdb74-636b-4785-b610-30b153cdf480 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:04.915136 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e0fbdb74-636b-4785-b610-30b153cdf480 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:04.916487 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e0fbdb74-636b-4785-b610-30b153cdf480 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:04.925292 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1271/2555] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:04.930121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d8962-3184-45a7-845a-a1efdf050c39 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:04.940055 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e32d8962-3184-45a7-845a-a1efdf050c39 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:04.944757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642c8e9-9001-4f4a-98f9-e50d9216d0e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:04.945471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642c8e9-9001-4f4a-98f9-e50d9216d0e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:04.946250 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642c8e9-9001-4f4a-98f9-e50d9216d0e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:04.946749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7642c8e9-9001-4f4a-98f9-e50d9216d0e8 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:04.947496 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642c8e9-9001-4f4a-98f9-e50d9216d0e8 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:17:04.947805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642c8e9-9001-4f4a-98f9-e50d9216d0e8 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:04.949401 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7642c8e9-9001-4f4a-98f9-e50d9216d0e8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:04.961788 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1285/2556] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:17:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:17:04.966041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbdb74-636b-4785-b610-30b153cdf480 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:04.975550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0fbdb74-636b-4785-b610-30b153cdf480 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:04.980371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772edd5a-f10b-4ff3-a440-352cfbf6db68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:04.981039 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772edd5a-f10b-4ff3-a440-352cfbf6db68 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:04.981322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772edd5a-f10b-4ff3-a440-352cfbf6db68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:04.981599 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772edd5a-f10b-4ff3-a440-352cfbf6db68 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:04.982333 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772edd5a-f10b-4ff3-a440-352cfbf6db68 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:04.984277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-772edd5a-f10b-4ff3-a440-352cfbf6db68 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:05.014014 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1272/2557] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:17:04 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:05.017491 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642c8e9-9001-4f4a-98f9-e50d9216d0e8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:05.027808 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7642c8e9-9001-4f4a-98f9-e50d9216d0e8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:05.087405 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083b517-754f-4504-935f-be334a357ad0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:05.088084 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083b517-754f-4504-935f-be334a357ad0 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:05.088373 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083b517-754f-4504-935f-be334a357ad0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:05.088655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5083b517-754f-4504-935f-be334a357ad0 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:05.089375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083b517-754f-4504-935f-be334a357ad0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:05.091341 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5083b517-754f-4504-935f-be334a357ad0 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:05.112774 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1286/2558] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:17:05 2022] POST /identity/v3/users => generated 294 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:05.116965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772edd5a-f10b-4ff3-a440-352cfbf6db68 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:05.128573 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772edd5a-f10b-4ff3-a440-352cfbf6db68 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:05.133335 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d7bfa-d0cc-48a6-b9b7-34774745aaef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:05.134006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d7bfa-d0cc-48a6-b9b7-34774745aaef admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:05.134297 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d7bfa-d0cc-48a6-b9b7-34774745aaef admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:05.134610 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681d7bfa-d0cc-48a6-b9b7-34774745aaef admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:05.135355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d7bfa-d0cc-48a6-b9b7-34774745aaef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:05.137153 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-681d7bfa-d0cc-48a6-b9b7-34774745aaef admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:05.148017 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1273/2559] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:05.153150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083b517-754f-4504-935f-be334a357ad0 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:05.166698 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5083b517-754f-4504-935f-be334a357ad0 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:05.171447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eacd43bb-40c9-41bb-aa05-74b39fc282fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:05.172120 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd43bb-40c9-41bb-aa05-74b39fc282fa admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:05.172399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd43bb-40c9-41bb-aa05-74b39fc282fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:05.172671 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eacd43bb-40c9-41bb-aa05-74b39fc282fa admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:05.173409 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd43bb-40c9-41bb-aa05-74b39fc282fa admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:05.175286 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eacd43bb-40c9-41bb-aa05-74b39fc282fa admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:05.186336 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1287/2560] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:05.190587 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d7bfa-d0cc-48a6-b9b7-34774745aaef admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:05.200792 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681d7bfa-d0cc-48a6-b9b7-34774745aaef admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:05.205476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a7dd2-d04a-41fd-b643-9c177eb567bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:05.206387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a7dd2-d04a-41fd-b643-9c177eb567bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:05.206676 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a7dd2-d04a-41fd-b643-9c177eb567bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:05.206956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63a7dd2-d04a-41fd-b643-9c177eb567bb admin admin] PATH_INFO: `/v3/projects/9755cba9955b4bdc9a838612e5c7c128/users/d935f03a80c0424a9da46b9dc8acf117/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:05.221985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a7dd2-d04a-41fd-b643-9c177eb567bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=9755cba9955b4bdc9a838612e5c7c128, user_id=d935f03a80c0424a9da46b9dc8acf117, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:05.224602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63a7dd2-d04a-41fd-b643-9c177eb567bb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:05.240739 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1274/2561] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:05 2022] PUT /identity/v3/projects/9755cba9955b4bdc9a838612e5c7c128/users/d935f03a80c0424a9da46b9dc8acf117/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:05.251740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed54cf-5811-476f-a96e-4269e73150a8 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:05.252029 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed54cf-5811-476f-a96e-4269e73150a8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:05.252309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ed54cf-5811-476f-a96e-4269e73150a8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:05.280875 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-b2ed54cf-5811-476f-a96e-4269e73150a8 None None] MFA Rules not processed for user `d935f03a80c0424a9da46b9dc8acf117`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:05.282091 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b2ed54cf-5811-476f-a96e-4269e73150a8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:05.350079 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1288/2562] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:05 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:05.355118 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a7dd2-d04a-41fd-b643-9c177eb567bb admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:05.399804 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:05.419223 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63a7dd2-d04a-41fd-b643-9c177eb567bb admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:05.424024 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38546d1d-5f14-4902-8de8-e0b38d8b4aa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:05.424717 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38546d1d-5f14-4902-8de8-e0b38d8b4aa3 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:05.425268 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38546d1d-5f14-4902-8de8-e0b38d8b4aa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:05.425550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38546d1d-5f14-4902-8de8-e0b38d8b4aa3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:05.426251 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38546d1d-5f14-4902-8de8-e0b38d8b4aa3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:05.428197 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38546d1d-5f14-4902-8de8-e0b38d8b4aa3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:05.454265 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1275/2563] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:17:05 2022] POST /identity/v3/projects => generated 387 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:05.458421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed54cf-5811-476f-a96e-4269e73150a8 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:05.470748 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ed54cf-5811-476f-a96e-4269e73150a8 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:05.502541 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:05.528100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5120b-f44e-4202-909e-dae8118b90ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:05.528774 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5120b-f44e-4202-909e-dae8118b90ce admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:05.529059 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5120b-f44e-4202-909e-dae8118b90ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:05.529340 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f5120b-f44e-4202-909e-dae8118b90ce admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:05.530038 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5120b-f44e-4202-909e-dae8118b90ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:05.532091 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94f5120b-f44e-4202-909e-dae8118b90ce admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:05.551447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1289/2564] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:17:05 2022] POST /identity/v3/users => generated 300 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:05.555870 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38546d1d-5f14-4902-8de8-e0b38d8b4aa3 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:05.565993 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38546d1d-5f14-4902-8de8-e0b38d8b4aa3 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:05.571345 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dcb06-b7b6-42a8-8596-5ab3dbdc1709 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:05.572005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dcb06-b7b6-42a8-8596-5ab3dbdc1709 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:05.572287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dcb06-b7b6-42a8-8596-5ab3dbdc1709 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:05.572562 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f77dcb06-b7b6-42a8-8596-5ab3dbdc1709 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:05.573315 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77dcb06-b7b6-42a8-8596-5ab3dbdc1709 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:05.574933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f77dcb06-b7b6-42a8-8596-5ab3dbdc1709 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:05.585343 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1276/2565] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:05.589497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5120b-f44e-4202-909e-dae8118b90ce admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:05.599710 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94f5120b-f44e-4202-909e-dae8118b90ce admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:05.604411 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c885c2-246e-4b5b-8c4e-3c0f1e3078ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:05.605294 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c885c2-246e-4b5b-8c4e-3c0f1e3078ff admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:05.605578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c885c2-246e-4b5b-8c4e-3c0f1e3078ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:05.605857 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37c885c2-246e-4b5b-8c4e-3c0f1e3078ff admin admin] PATH_INFO: `/v3/projects/c22bca9f30334224afbb372cbe227bc6/users/d008350e00484bca92d62addee04eba5/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:05.622046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c885c2-246e-4b5b-8c4e-3c0f1e3078ff admin admin] RBAC: Authorizing `identity:create_grant(project_id=c22bca9f30334224afbb372cbe227bc6, user_id=d008350e00484bca92d62addee04eba5, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:05.624757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37c885c2-246e-4b5b-8c4e-3c0f1e3078ff admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:05.638023 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1290/2566] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:05 2022] PUT /identity/v3/projects/c22bca9f30334224afbb372cbe227bc6/users/d008350e00484bca92d62addee04eba5/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:17:05.642121 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dcb06-b7b6-42a8-8596-5ab3dbdc1709 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:05.652691 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f77dcb06-b7b6-42a8-8596-5ab3dbdc1709 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:05.709476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55603d7a-4a57-41e4-9dac-900c2dfa3458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:05.710205 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55603d7a-4a57-41e4-9dac-900c2dfa3458 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:05.710492 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55603d7a-4a57-41e4-9dac-900c2dfa3458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:05.710777 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55603d7a-4a57-41e4-9dac-900c2dfa3458 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:05.712005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55603d7a-4a57-41e4-9dac-900c2dfa3458 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:05.713611 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55603d7a-4a57-41e4-9dac-900c2dfa3458 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:05.723785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1277/2567] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:05 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:05.728745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c885c2-246e-4b5b-8c4e-3c0f1e3078ff admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:05.738902 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37c885c2-246e-4b5b-8c4e-3c0f1e3078ff admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:05.743656 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc09ac-327d-4bdc-b439-62805a0f3db0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:05.744564 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc09ac-327d-4bdc-b439-62805a0f3db0 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:05.744829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc09ac-327d-4bdc-b439-62805a0f3db0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:05.745103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0fc09ac-327d-4bdc-b439-62805a0f3db0 admin admin] PATH_INFO: `/v3/projects/c22bca9f30334224afbb372cbe227bc6/users/d008350e00484bca92d62addee04eba5/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:05.747080 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc09ac-327d-4bdc-b439-62805a0f3db0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c22bca9f30334224afbb372cbe227bc6, user_id=d008350e00484bca92d62addee04eba5, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:05.749887 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0fc09ac-327d-4bdc-b439-62805a0f3db0 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:05.761324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1291/2568] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:05 2022] PUT /identity/v3/projects/c22bca9f30334224afbb372cbe227bc6/users/d008350e00484bca92d62addee04eba5/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:05.772595 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c49918-4ac2-4c69-bfad-1d35817c3959 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:05.772889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c49918-4ac2-4c69-bfad-1d35817c3959 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:05.773175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c49918-4ac2-4c69-bfad-1d35817c3959 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:05.800908 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-65c49918-4ac2-4c69-bfad-1d35817c3959 None None] MFA Rules not processed for user `d008350e00484bca92d62addee04eba5`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:05.802125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-65c49918-4ac2-4c69-bfad-1d35817c3959 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:05.869334 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1278/2569] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:05.881135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc09ac-327d-4bdc-b439-62805a0f3db0 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:05.883411 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d0fc09ac-327d-4bdc-b439-62805a0f3db0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:05.943884 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0fc09ac-327d-4bdc-b439-62805a0f3db0 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:05.948633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cce2f5a-b126-4373-bc3a-a774924c304c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:05.949525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce2f5a-b126-4373-bc3a-a774924c304c service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:05.949874 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce2f5a-b126-4373-bc3a-a774924c304c service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:05.950109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cce2f5a-b126-4373-bc3a-a774924c304c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:05.960718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce2f5a-b126-4373-bc3a-a774924c304c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:05.962822 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cce2f5a-b126-4373-bc3a-a774924c304c service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:05.972665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1292/2570] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:06.553005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c49918-4ac2-4c69-bfad-1d35817c3959 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:06.595304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:06.619231 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c49918-4ac2-4c69-bfad-1d35817c3959 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:06.623902 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a23244-f972-42b9-a0b3-574689bf783c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:06.624774 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a23244-f972-42b9-a0b3-574689bf783c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:06.625058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a23244-f972-42b9-a0b3-574689bf783c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:06.625338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a23244-f972-42b9-a0b3-574689bf783c admin admin] PATH_INFO: `/v3/users/d935f03a80c0424a9da46b9dc8acf117` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:06.626642 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a23244-f972-42b9-a0b3-574689bf783c admin admin] RBAC: Authorizing `identity:delete_user(user_id=d935f03a80c0424a9da46b9dc8acf117)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:06.628575 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0a23244-f972-42b9-a0b3-574689bf783c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:06.679499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b0a23244-f972-42b9-a0b3-574689bf783c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd935f03a80c0424a9da46b9dc8acf117'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:06.680725 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b0a23244-f972-42b9-a0b3-574689bf783c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd935f03a80c0424a9da46b9dc8acf117'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:06.688503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b0a23244-f972-42b9-a0b3-574689bf783c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd935f03a80c0424a9da46b9dc8acf117'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:06.689231 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b0a23244-f972-42b9-a0b3-574689bf783c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd935f03a80c0424a9da46b9dc8acf117'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:06.701896 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-b0a23244-f972-42b9-a0b3-574689bf783c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd935f03a80c0424a9da46b9dc8acf117'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:06.714143 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1279/2571] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:17:06 2022] DELETE /identity/v3/users/d935f03a80c0424a9da46b9dc8acf117 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:06.718789 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ee53e-f259-4410-b6d1-9ec28babd94c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:06.719209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ee53e-f259-4410-b6d1-9ec28babd94c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:06.719337 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47ee53e-f259-4410-b6d1-9ec28babd94c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:06.736756 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c47ee53e-f259-4410-b6d1-9ec28babd94c None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:06.737995 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c47ee53e-f259-4410-b6d1-9ec28babd94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:06.799749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1293/2572] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:06.810235 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a23244-f972-42b9-a0b3-574689bf783c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:06.811718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b0a23244-f972-42b9-a0b3-574689bf783c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:06.871291 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a23244-f972-42b9-a0b3-574689bf783c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:06.875218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957887f-b196-46f1-816c-02268ca2f555 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:06.876010 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957887f-b196-46f1-816c-02268ca2f555 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:06.876328 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957887f-b196-46f1-816c-02268ca2f555 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:06.876701 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1957887f-b196-46f1-816c-02268ca2f555 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:06.886730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957887f-b196-46f1-816c-02268ca2f555 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:06.888170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1957887f-b196-46f1-816c-02268ca2f555 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:06.904804 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1280/2573] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:07.203899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ee53e-f259-4410-b6d1-9ec28babd94c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:07.205326 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c47ee53e-f259-4410-b6d1-9ec28babd94c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:07.220210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47ee53e-f259-4410-b6d1-9ec28babd94c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:07.225637 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6a625-fd1e-4f47-8d01-6480f50b0e5e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:07.227119 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6a625-fd1e-4f47-8d01-6480f50b0e5e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:07.227394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6a625-fd1e-4f47-8d01-6480f50b0e5e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:07.227674 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf6a625-fd1e-4f47-8d01-6480f50b0e5e admin admin] PATH_INFO: `/v3/projects/9755cba9955b4bdc9a838612e5c7c128` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:07.229161 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6a625-fd1e-4f47-8d01-6480f50b0e5e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9755cba9955b4bdc9a838612e5c7c128)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:07.231336 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf6a625-fd1e-4f47-8d01-6480f50b0e5e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:07.262587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:07.280448 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9bf6a625-fd1e-4f47-8d01-6480f50b0e5e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9755cba9955b4bdc9a838612e5c7c128', 'request_id': 'req-9bf6a625-fd1e-4f47-8d01-6480f50b0e5e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:07.287914 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9bf6a625-fd1e-4f47-8d01-6480f50b0e5e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9755cba9955b4bdc9a838612e5c7c128', 'request_id': 'req-9bf6a625-fd1e-4f47-8d01-6480f50b0e5e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:07.294121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9bf6a625-fd1e-4f47-8d01-6480f50b0e5e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9755cba9955b4bdc9a838612e5c7c128', 'request_id': 'req-9bf6a625-fd1e-4f47-8d01-6480f50b0e5e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:07.305616 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9bf6a625-fd1e-4f47-8d01-6480f50b0e5e admin admin] The token cache is being invalidate because project 9755cba9955b4bdc9a838612e5c7c128 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:17:07.305843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9bf6a625-fd1e-4f47-8d01-6480f50b0e5e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:07.306528 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-9bf6a625-fd1e-4f47-8d01-6480f50b0e5e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:07.308079 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1294/2574] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:17:07 2022] DELETE /identity/v3/projects/9755cba9955b4bdc9a838612e5c7c128 => generated 0 bytes in 105 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:07.312009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957887f-b196-46f1-816c-02268ca2f555 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:07.323387 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1957887f-b196-46f1-816c-02268ca2f555 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:07.378792 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22e26a5c-def7-46d5-8aac-3160423da720 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:07.379657 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e26a5c-def7-46d5-8aac-3160423da720 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:07.379945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e26a5c-def7-46d5-8aac-3160423da720 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:07.380225 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22e26a5c-def7-46d5-8aac-3160423da720 admin admin] PATH_INFO: `/v3/users/d008350e00484bca92d62addee04eba5` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:07.381519 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e26a5c-def7-46d5-8aac-3160423da720 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d008350e00484bca92d62addee04eba5)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:07.383993 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22e26a5c-def7-46d5-8aac-3160423da720 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:07.385508 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:07.431234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-22e26a5c-def7-46d5-8aac-3160423da720 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd008350e00484bca92d62addee04eba5'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:07.431893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-22e26a5c-def7-46d5-8aac-3160423da720 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd008350e00484bca92d62addee04eba5'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:07.439127 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-22e26a5c-def7-46d5-8aac-3160423da720 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd008350e00484bca92d62addee04eba5'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:07.439703 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-22e26a5c-def7-46d5-8aac-3160423da720 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd008350e00484bca92d62addee04eba5'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:07.452161 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-22e26a5c-def7-46d5-8aac-3160423da720 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd008350e00484bca92d62addee04eba5'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:07.465232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1281/2575] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:17:07 2022] DELETE /identity/v3/users/d008350e00484bca92d62addee04eba5 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:07.745391 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6a625-fd1e-4f47-8d01-6480f50b0e5e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:07.746993 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9bf6a625-fd1e-4f47-8d01-6480f50b0e5e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:07.761931 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf6a625-fd1e-4f47-8d01-6480f50b0e5e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:07.818590 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00a54db4-07c7-4fcd-8e00-c37ce78eb359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:07.819510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a54db4-07c7-4fcd-8e00-c37ce78eb359 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:07.819732 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a54db4-07c7-4fcd-8e00-c37ce78eb359 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:07.820005 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00a54db4-07c7-4fcd-8e00-c37ce78eb359 admin admin] PATH_INFO: `/v3/projects/c22bca9f30334224afbb372cbe227bc6` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:07.822073 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a54db4-07c7-4fcd-8e00-c37ce78eb359 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c22bca9f30334224afbb372cbe227bc6)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:07.824009 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00a54db4-07c7-4fcd-8e00-c37ce78eb359 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:07.872619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-00a54db4-07c7-4fcd-8e00-c37ce78eb359 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c22bca9f30334224afbb372cbe227bc6', 'request_id': 'req-00a54db4-07c7-4fcd-8e00-c37ce78eb359'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:07.879859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-00a54db4-07c7-4fcd-8e00-c37ce78eb359 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c22bca9f30334224afbb372cbe227bc6', 'request_id': 'req-00a54db4-07c7-4fcd-8e00-c37ce78eb359'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:07.886411 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-00a54db4-07c7-4fcd-8e00-c37ce78eb359 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c22bca9f30334224afbb372cbe227bc6', 'request_id': 'req-00a54db4-07c7-4fcd-8e00-c37ce78eb359'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:07.897508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-00a54db4-07c7-4fcd-8e00-c37ce78eb359 admin admin] The token cache is being invalidate because project c22bca9f30334224afbb372cbe227bc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:17:07.897791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-00a54db4-07c7-4fcd-8e00-c37ce78eb359 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:07.898542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-00a54db4-07c7-4fcd-8e00-c37ce78eb359 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:07.900478 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1295/2576] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:17:07 2022] DELETE /identity/v3/projects/c22bca9f30334224afbb372cbe227bc6 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:07.905917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c052df-4c12-456f-a523-6b92bd56c62e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:07.906472 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c052df-4c12-456f-a523-6b92bd56c62e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:07.906558 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c052df-4c12-456f-a523-6b92bd56c62e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:07.925654 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-77c052df-4c12-456f-a523-6b92bd56c62e None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:07.926898 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-77c052df-4c12-456f-a523-6b92bd56c62e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:07.986946 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1282/2577] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:07.995561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc596d27-a14a-46cc-9440-85296b46ae0d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:07.995853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc596d27-a14a-46cc-9440-85296b46ae0d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:07.996135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc596d27-a14a-46cc-9440-85296b46ae0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:08.014834 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-dc596d27-a14a-46cc-9440-85296b46ae0d None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:08.016051 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dc596d27-a14a-46cc-9440-85296b46ae0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:08.024056 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1296/2578] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:08.028321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c052df-4c12-456f-a523-6b92bd56c62e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:08.037976 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c052df-4c12-456f-a523-6b92bd56c62e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:08.042998 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23486185-3d12-475b-90fb-52f64a7ebf77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:08.043718 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23486185-3d12-475b-90fb-52f64a7ebf77 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:08.044014 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23486185-3d12-475b-90fb-52f64a7ebf77 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:08.044290 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23486185-3d12-475b-90fb-52f64a7ebf77 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:08.045061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23486185-3d12-475b-90fb-52f64a7ebf77 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:17:08.045378 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23486185-3d12-475b-90fb-52f64a7ebf77 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:08.047070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23486185-3d12-475b-90fb-52f64a7ebf77 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:08.059834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1283/2579] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:17:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:17:08.064224 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc596d27-a14a-46cc-9440-85296b46ae0d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:08.074248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc596d27-a14a-46cc-9440-85296b46ae0d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:08.078929 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb9f8b-4ff0-4168-9ed1-731c17748aac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:08.079588 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb9f8b-4ff0-4168-9ed1-731c17748aac admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:08.079875 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb9f8b-4ff0-4168-9ed1-731c17748aac admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:08.080156 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddbb9f8b-4ff0-4168-9ed1-731c17748aac admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:08.080850 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb9f8b-4ff0-4168-9ed1-731c17748aac admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:08.082785 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddbb9f8b-4ff0-4168-9ed1-731c17748aac admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:08.111030 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1297/2580] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:17:08 2022] POST /identity/v3/projects => generated 371 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:08.114597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23486185-3d12-475b-90fb-52f64a7ebf77 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:08.125797 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23486185-3d12-475b-90fb-52f64a7ebf77 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:08.186533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d9f75-902c-44fd-ac66-4416399df1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:08.187211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d9f75-902c-44fd-ac66-4416399df1b9 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:08.187496 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d9f75-902c-44fd-ac66-4416399df1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:08.187776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4d9f75-902c-44fd-ac66-4416399df1b9 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:08.188490 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d9f75-902c-44fd-ac66-4416399df1b9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:08.190424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4d9f75-902c-44fd-ac66-4416399df1b9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:08.200234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=86,keystone:DELETE=10,keystone:INSERT=4 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:08.212716 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1284/2581] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:17:08 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:08.216815 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb9f8b-4ff0-4168-9ed1-731c17748aac admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:08.227687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddbb9f8b-4ff0-4168-9ed1-731c17748aac admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:08.232685 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a93c9-2660-4ae8-a729-75eacdeb42a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:08.233356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a93c9-2660-4ae8-a729-75eacdeb42a9 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:08.233653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a93c9-2660-4ae8-a729-75eacdeb42a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:08.233932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e7a93c9-2660-4ae8-a729-75eacdeb42a9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:08.234697 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a93c9-2660-4ae8-a729-75eacdeb42a9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:08.236288 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e7a93c9-2660-4ae8-a729-75eacdeb42a9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:08.247733 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1298/2582] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:08.251752 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d9f75-902c-44fd-ac66-4416399df1b9 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:08.261931 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4d9f75-902c-44fd-ac66-4416399df1b9 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:08.268570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ac5cee-1838-4af7-aeae-7a4c6a8e3499 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:08.269259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac5cee-1838-4af7-aeae-7a4c6a8e3499 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:08.269565 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac5cee-1838-4af7-aeae-7a4c6a8e3499 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:08.269853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ac5cee-1838-4af7-aeae-7a4c6a8e3499 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:08.270625 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac5cee-1838-4af7-aeae-7a4c6a8e3499 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:08.272403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33ac5cee-1838-4af7-aeae-7a4c6a8e3499 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:08.284547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1285/2583] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:08 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:08.288832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a93c9-2660-4ae8-a729-75eacdeb42a9 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:08.296207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:08.301644 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e7a93c9-2660-4ae8-a729-75eacdeb42a9 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:08.308778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51affb6f-d4e7-493e-94c9-46bf132c7efe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:08.309910 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51affb6f-d4e7-493e-94c9-46bf132c7efe admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:08.310520 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51affb6f-d4e7-493e-94c9-46bf132c7efe admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:08.310975 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51affb6f-d4e7-493e-94c9-46bf132c7efe admin admin] PATH_INFO: `/v3/projects/073cf8e213cf49daac224cf262d1b249/users/79c1e8ceaa66420e93724d271c7c4290/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:08.326478 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51affb6f-d4e7-493e-94c9-46bf132c7efe admin admin] RBAC: Authorizing `identity:create_grant(project_id=073cf8e213cf49daac224cf262d1b249, user_id=79c1e8ceaa66420e93724d271c7c4290, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:08.329061 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51affb6f-d4e7-493e-94c9-46bf132c7efe admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:08.343676 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1299/2584] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:08 2022] PUT /identity/v3/projects/073cf8e213cf49daac224cf262d1b249/users/79c1e8ceaa66420e93724d271c7c4290/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:17:08.348911 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b2233-49d5-4f7e-8cb6-f468aeb7eaac None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:08.349332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b2233-49d5-4f7e-8cb6-f468aeb7eaac None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:08.349508 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0b2233-49d5-4f7e-8cb6-f468aeb7eaac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:08.369737 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-bf0b2233-49d5-4f7e-8cb6-f468aeb7eaac None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:08.371504 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bf0b2233-49d5-4f7e-8cb6-f468aeb7eaac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:08.437220 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1286/2585] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:08.447590 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51affb6f-d4e7-493e-94c9-46bf132c7efe admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:08.449448 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-51affb6f-d4e7-493e-94c9-46bf132c7efe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:08.512034 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51affb6f-d4e7-493e-94c9-46bf132c7efe admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:08.515732 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe35a2-5fc1-46d3-a952-860ee1fe1a40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:08.516382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe35a2-5fc1-46d3-a952-860ee1fe1a40 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:08.516670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe35a2-5fc1-46d3-a952-860ee1fe1a40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:08.516946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2fe35a2-5fc1-46d3-a952-860ee1fe1a40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:08.527328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe35a2-5fc1-46d3-a952-860ee1fe1a40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:08.528766 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2fe35a2-5fc1-46d3-a952-860ee1fe1a40 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:08.539002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1300/2586] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:09.145171 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac3773-46fc-4374-ae8f-d5b683f7df1c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:09.145677 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac3773-46fc-4374-ae8f-d5b683f7df1c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:09.145796 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ac3773-46fc-4374-ae8f-d5b683f7df1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:09.177465 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-24ac3773-46fc-4374-ae8f-d5b683f7df1c None None] MFA Rules not processed for user `79c1e8ceaa66420e93724d271c7c4290`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:09.178732 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-24ac3773-46fc-4374-ae8f-d5b683f7df1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:09.247517 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1287/2587] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:09 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:09.252085 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe35a2-5fc1-46d3-a952-860ee1fe1a40 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:09.261785 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2fe35a2-5fc1-46d3-a952-860ee1fe1a40 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:09.266555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763172fa-d999-49d6-8235-4f318b7721b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:09.267215 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763172fa-d999-49d6-8235-4f318b7721b4 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:09.267501 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763172fa-d999-49d6-8235-4f318b7721b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:09.267786 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763172fa-d999-49d6-8235-4f318b7721b4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:09.268521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763172fa-d999-49d6-8235-4f318b7721b4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:09.270475 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763172fa-d999-49d6-8235-4f318b7721b4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:09.296536 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1301/2588] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:17:09 2022] POST /identity/v3/projects => generated 371 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:09.300896 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac3773-46fc-4374-ae8f-d5b683f7df1c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:09.314490 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ac3773-46fc-4374-ae8f-d5b683f7df1c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:09.373556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9083414e-3081-454f-8ad6-96552922077a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:09.374153 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083414e-3081-454f-8ad6-96552922077a admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:09.374479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083414e-3081-454f-8ad6-96552922077a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:09.374760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9083414e-3081-454f-8ad6-96552922077a admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:09.375464 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9083414e-3081-454f-8ad6-96552922077a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:09.377371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9083414e-3081-454f-8ad6-96552922077a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:09.396743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1288/2589] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:17:09 2022] POST /identity/v3/users => generated 292 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:17:09.401205 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763172fa-d999-49d6-8235-4f318b7721b4 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:09.411503 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763172fa-d999-49d6-8235-4f318b7721b4 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:09.416739 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c10c2d-c1d8-45a5-9b1d-1f7dc2b65bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:09.417346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c10c2d-c1d8-45a5-9b1d-1f7dc2b65bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:09.417631 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c10c2d-c1d8-45a5-9b1d-1f7dc2b65bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:09.417908 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62c10c2d-c1d8-45a5-9b1d-1f7dc2b65bf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:09.418682 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c10c2d-c1d8-45a5-9b1d-1f7dc2b65bf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:09.420272 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62c10c2d-c1d8-45a5-9b1d-1f7dc2b65bf9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:09.430901 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1302/2590] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:09.434978 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9083414e-3081-454f-8ad6-96552922077a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:09.441232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:09.444981 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9083414e-3081-454f-8ad6-96552922077a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:09.451225 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d8669-1f50-49b0-9f4e-660ec24c360b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:09.452117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d8669-1f50-49b0-9f4e-660ec24c360b admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:09.452403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d8669-1f50-49b0-9f4e-660ec24c360b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:09.452683 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4d8669-1f50-49b0-9f4e-660ec24c360b admin admin] PATH_INFO: `/v3/projects/b50a0de22c844f1e9be990ff1f6ed771/users/6cc94599b4974433959a5ab05a217d9c/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:09.469058 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d8669-1f50-49b0-9f4e-660ec24c360b admin admin] RBAC: Authorizing `identity:create_grant(project_id=b50a0de22c844f1e9be990ff1f6ed771, user_id=6cc94599b4974433959a5ab05a217d9c, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:09.471745 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4d8669-1f50-49b0-9f4e-660ec24c360b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:09.484812 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1289/2591] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:09 2022] PUT /identity/v3/projects/b50a0de22c844f1e9be990ff1f6ed771/users/6cc94599b4974433959a5ab05a217d9c/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:17:09.489335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c10c2d-c1d8-45a5-9b1d-1f7dc2b65bf9 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:09.499829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62c10c2d-c1d8-45a5-9b1d-1f7dc2b65bf9 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:09.557734 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90f71bd8-c5b1-4959-a8dc-c351e1e36721 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:09.558431 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f71bd8-c5b1-4959-a8dc-c351e1e36721 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:09.558716 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f71bd8-c5b1-4959-a8dc-c351e1e36721 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:09.558987 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90f71bd8-c5b1-4959-a8dc-c351e1e36721 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:09.559710 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f71bd8-c5b1-4959-a8dc-c351e1e36721 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:09.561272 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90f71bd8-c5b1-4959-a8dc-c351e1e36721 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:09.571499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1303/2592] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:09 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:09.575559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d8669-1f50-49b0-9f4e-660ec24c360b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:09.586070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4d8669-1f50-49b0-9f4e-660ec24c360b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:09.591093 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17a416-192b-4772-bfa2-0c66b4965334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:09.591976 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17a416-192b-4772-bfa2-0c66b4965334 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:09.592261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17a416-192b-4772-bfa2-0c66b4965334 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:09.592536 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd17a416-192b-4772-bfa2-0c66b4965334 admin admin] PATH_INFO: `/v3/projects/b50a0de22c844f1e9be990ff1f6ed771/users/6cc94599b4974433959a5ab05a217d9c/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:09.594595 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17a416-192b-4772-bfa2-0c66b4965334 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b50a0de22c844f1e9be990ff1f6ed771, user_id=6cc94599b4974433959a5ab05a217d9c, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:09.597271 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd17a416-192b-4772-bfa2-0c66b4965334 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:09.608476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1290/2593] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:09 2022] PUT /identity/v3/projects/b50a0de22c844f1e9be990ff1f6ed771/users/6cc94599b4974433959a5ab05a217d9c/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:10.208402 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa55d50-cf53-42ca-952f-8e1b88456a92 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:10.208700 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa55d50-cf53-42ca-952f-8e1b88456a92 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:10.208983 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa55d50-cf53-42ca-952f-8e1b88456a92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:10.236115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-bfa55d50-cf53-42ca-952f-8e1b88456a92 None None] MFA Rules not processed for user `6cc94599b4974433959a5ab05a217d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:10.237336 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bfa55d50-cf53-42ca-952f-8e1b88456a92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:10.271811 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:10.309777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1304/2594] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:10 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:10.329475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17a416-192b-4772-bfa2-0c66b4965334 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:10.331223 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bd17a416-192b-4772-bfa2-0c66b4965334 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:10.389473 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd17a416-192b-4772-bfa2-0c66b4965334 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:10.392619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870b925-84cb-42eb-ac41-1472d4f3bc6e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:10.393270 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870b925-84cb-42eb-ac41-1472d4f3bc6e service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:10.393556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870b925-84cb-42eb-ac41-1472d4f3bc6e service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:10.393861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d870b925-84cb-42eb-ac41-1472d4f3bc6e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:10.449782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870b925-84cb-42eb-ac41-1472d4f3bc6e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:10.451201 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d870b925-84cb-42eb-ac41-1472d4f3bc6e service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:10.460161 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1291/2595] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:10 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:11.805310 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa55d50-cf53-42ca-952f-8e1b88456a92 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:11.815031 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa55d50-cf53-42ca-952f-8e1b88456a92 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:11.818608 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202dbf0-fac3-41dc-bf64-093b08dbf951 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:11.819246 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202dbf0-fac3-41dc-bf64-093b08dbf951 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:11.819529 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202dbf0-fac3-41dc-bf64-093b08dbf951 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:11.819801 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2202dbf0-fac3-41dc-bf64-093b08dbf951 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:11.829097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202dbf0-fac3-41dc-bf64-093b08dbf951 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:11.830556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2202dbf0-fac3-41dc-bf64-093b08dbf951 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:11.840407 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1305/2596] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:11 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:16.094694 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870b925-84cb-42eb-ac41-1472d4f3bc6e service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:16.096130 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d870b925-84cb-42eb-ac41-1472d4f3bc6e service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:16.108412 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d870b925-84cb-42eb-ac41-1472d4f3bc6e service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:16.112131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab6281-0de2-4351-b178-fe07302ed8d1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:16.112965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab6281-0de2-4351-b178-fe07302ed8d1 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:16.113074 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab6281-0de2-4351-b178-fe07302ed8d1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:16.113596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ab6281-0de2-4351-b178-fe07302ed8d1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:16.115225 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-36ab6281-0de2-4351-b178-fe07302ed8d1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:16.186883 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab6281-0de2-4351-b178-fe07302ed8d1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:16.188314 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ab6281-0de2-4351-b178-fe07302ed8d1 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:16.199304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1292/2597] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:21.836184 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=19 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:24.758291 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202dbf0-fac3-41dc-bf64-093b08dbf951 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:24.821870 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2202dbf0-fac3-41dc-bf64-093b08dbf951 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:24.827202 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc32d2e-f850-47a0-a9e7-b7cb8d72fab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:24.828154 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc32d2e-f850-47a0-a9e7-b7cb8d72fab5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:24.828435 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc32d2e-f850-47a0-a9e7-b7cb8d72fab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:24.828710 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bc32d2e-f850-47a0-a9e7-b7cb8d72fab5 admin admin] PATH_INFO: `/v3/users/79c1e8ceaa66420e93724d271c7c4290` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:24.829890 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc32d2e-f850-47a0-a9e7-b7cb8d72fab5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=79c1e8ceaa66420e93724d271c7c4290)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:24.831836 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bc32d2e-f850-47a0-a9e7-b7cb8d72fab5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:24.878917 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4bc32d2e-f850-47a0-a9e7-b7cb8d72fab5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '79c1e8ceaa66420e93724d271c7c4290'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:24.893060 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4bc32d2e-f850-47a0-a9e7-b7cb8d72fab5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '79c1e8ceaa66420e93724d271c7c4290'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:24.900465 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4bc32d2e-f850-47a0-a9e7-b7cb8d72fab5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '79c1e8ceaa66420e93724d271c7c4290'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:24.901037 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4bc32d2e-f850-47a0-a9e7-b7cb8d72fab5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '79c1e8ceaa66420e93724d271c7c4290'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:24.908584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4bc32d2e-f850-47a0-a9e7-b7cb8d72fab5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '79c1e8ceaa66420e93724d271c7c4290'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:24.915195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1306/2598] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:17:24 2022] DELETE /identity/v3/users/79c1e8ceaa66420e93724d271c7c4290 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:25.086788 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab6281-0de2-4351-b178-fe07302ed8d1 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:25.098574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ab6281-0de2-4351-b178-fe07302ed8d1 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:25.103254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:25.160001 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8be18-2c23-410f-b55f-3f3d50088a50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:25.160866 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8be18-2c23-410f-b55f-3f3d50088a50 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:25.161142 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8be18-2c23-410f-b55f-3f3d50088a50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:25.161413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c8be18-2c23-410f-b55f-3f3d50088a50 admin admin] PATH_INFO: `/v3/projects/073cf8e213cf49daac224cf262d1b249` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:25.162626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8be18-2c23-410f-b55f-3f3d50088a50 admin admin] RBAC: Authorizing `identity:delete_project(project_id=073cf8e213cf49daac224cf262d1b249)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:25.165117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c8be18-2c23-410f-b55f-3f3d50088a50 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:25.213322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-04c8be18-2c23-410f-b55f-3f3d50088a50 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '073cf8e213cf49daac224cf262d1b249', 'request_id': 'req-04c8be18-2c23-410f-b55f-3f3d50088a50'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:25.218865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-04c8be18-2c23-410f-b55f-3f3d50088a50 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '073cf8e213cf49daac224cf262d1b249', 'request_id': 'req-04c8be18-2c23-410f-b55f-3f3d50088a50'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:25.227303 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-04c8be18-2c23-410f-b55f-3f3d50088a50 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '073cf8e213cf49daac224cf262d1b249', 'request_id': 'req-04c8be18-2c23-410f-b55f-3f3d50088a50'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:25.238278 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-04c8be18-2c23-410f-b55f-3f3d50088a50 admin admin] The token cache is being invalidate because project 073cf8e213cf49daac224cf262d1b249 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:17:25.238596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-04c8be18-2c23-410f-b55f-3f3d50088a50 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:25.239228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-04c8be18-2c23-410f-b55f-3f3d50088a50 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:25.240959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1293/2599] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:17:25 2022] DELETE /identity/v3/projects/073cf8e213cf49daac224cf262d1b249 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:25.244851 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc32d2e-f850-47a0-a9e7-b7cb8d72fab5 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:25.246265 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4bc32d2e-f850-47a0-a9e7-b7cb8d72fab5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:25.259428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bc32d2e-f850-47a0-a9e7-b7cb8d72fab5 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:25.317309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63bb1f-c774-455b-a689-8309461deec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:25.318105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63bb1f-c774-455b-a689-8309461deec5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:25.318424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63bb1f-c774-455b-a689-8309461deec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:25.318704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa63bb1f-c774-455b-a689-8309461deec5 admin admin] PATH_INFO: `/v3/users/6cc94599b4974433959a5ab05a217d9c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:25.320411 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa63bb1f-c774-455b-a689-8309461deec5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cc94599b4974433959a5ab05a217d9c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:25.322365 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa63bb1f-c774-455b-a689-8309461deec5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:25.345606 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:25.372230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-fa63bb1f-c774-455b-a689-8309461deec5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cc94599b4974433959a5ab05a217d9c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:25.385370 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-fa63bb1f-c774-455b-a689-8309461deec5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cc94599b4974433959a5ab05a217d9c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:25.392542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-fa63bb1f-c774-455b-a689-8309461deec5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cc94599b4974433959a5ab05a217d9c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:25.393248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-fa63bb1f-c774-455b-a689-8309461deec5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cc94599b4974433959a5ab05a217d9c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:25.400130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-fa63bb1f-c774-455b-a689-8309461deec5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cc94599b4974433959a5ab05a217d9c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:25.406120 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1307/2600] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:17:25 2022] DELETE /identity/v3/users/6cc94599b4974433959a5ab05a217d9c => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:25.577320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8be18-2c23-410f-b55f-3f3d50088a50 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:25.578834 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-04c8be18-2c23-410f-b55f-3f3d50088a50 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:25.593470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c8be18-2c23-410f-b55f-3f3d50088a50 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:25.668649 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-523c419b-ff46-4655-bf34-d7a51eee90f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:25.668649 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c419b-ff46-4655-bf34-d7a51eee90f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:25.668649 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c419b-ff46-4655-bf34-d7a51eee90f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:25.668649 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523c419b-ff46-4655-bf34-d7a51eee90f0 admin admin] PATH_INFO: `/v3/projects/b50a0de22c844f1e9be990ff1f6ed771` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:25.671158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c419b-ff46-4655-bf34-d7a51eee90f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b50a0de22c844f1e9be990ff1f6ed771)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:25.672923 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-523c419b-ff46-4655-bf34-d7a51eee90f0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:25.676773 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63bb1f-c774-455b-a689-8309461deec5 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:25.678233 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-fa63bb1f-c774-455b-a689-8309461deec5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:25.717471 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:25.726858 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-523c419b-ff46-4655-bf34-d7a51eee90f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b50a0de22c844f1e9be990ff1f6ed771', 'request_id': 'req-523c419b-ff46-4655-bf34-d7a51eee90f0'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:25.733423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-523c419b-ff46-4655-bf34-d7a51eee90f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b50a0de22c844f1e9be990ff1f6ed771', 'request_id': 'req-523c419b-ff46-4655-bf34-d7a51eee90f0'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:25.741078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa63bb1f-c774-455b-a689-8309461deec5 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:25.741986 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-523c419b-ff46-4655-bf34-d7a51eee90f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b50a0de22c844f1e9be990ff1f6ed771', 'request_id': 'req-523c419b-ff46-4655-bf34-d7a51eee90f0'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:25.754994 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-523c419b-ff46-4655-bf34-d7a51eee90f0 admin admin] The token cache is being invalidate because project b50a0de22c844f1e9be990ff1f6ed771 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:17:25.755601 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-523c419b-ff46-4655-bf34-d7a51eee90f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:25.756277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-523c419b-ff46-4655-bf34-d7a51eee90f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:25.757894 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1294/2601] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:17:25 2022] DELETE /identity/v3/projects/b50a0de22c844f1e9be990ff1f6ed771 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:25.762800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeecc72-57d0-4336-b09b-8af482c2d7ed None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:25.763099 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeecc72-57d0-4336-b09b-8af482c2d7ed None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:25.763372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aaeecc72-57d0-4336-b09b-8af482c2d7ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:25.783007 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-aaeecc72-57d0-4336-b09b-8af482c2d7ed None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:25.784252 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-aaeecc72-57d0-4336-b09b-8af482c2d7ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:25.794682 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb67a60-e47b-40af-b957-da9161e25b7e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:25.795287 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb67a60-e47b-40af-b957-da9161e25b7e service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:25.795563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb67a60-e47b-40af-b957-da9161e25b7e service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:25.795839 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb67a60-e47b-40af-b957-da9161e25b7e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:25.797645 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0fb67a60-e47b-40af-b957-da9161e25b7e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:25.845330 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1295/2602] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:17:25.853330 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812a28a-9296-4872-8d58-5297ae31e830 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:25.853628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812a28a-9296-4872-8d58-5297ae31e830 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:25.853913 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d812a28a-9296-4872-8d58-5297ae31e830 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:25.861445 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb67a60-e47b-40af-b957-da9161e25b7e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:25.862932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb67a60-e47b-40af-b957-da9161e25b7e service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:25.871693 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d812a28a-9296-4872-8d58-5297ae31e830 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:25.872898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1308/2603] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:25.873014 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d812a28a-9296-4872-8d58-5297ae31e830 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:25.882149 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1296/2604] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:17:25.888222 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb67a60-e47b-40af-b957-da9161e25b7e service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:25.895203 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=7,keystone:SELECT=92,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:25.899534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb67a60-e47b-40af-b957-da9161e25b7e service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:25.905876 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ca9ea-0748-4e8c-9052-2e56590e1790 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:25.906648 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ca9ea-0748-4e8c-9052-2e56590e1790 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:25.906867 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ca9ea-0748-4e8c-9052-2e56590e1790 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:25.907405 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa4ca9ea-0748-4e8c-9052-2e56590e1790 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:25.908291 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ca9ea-0748-4e8c-9052-2e56590e1790 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:17:25.908864 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ca9ea-0748-4e8c-9052-2e56590e1790 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:25.910509 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa4ca9ea-0748-4e8c-9052-2e56590e1790 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:25.928075 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1309/2605] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:17:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 14:17:25.929505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812a28a-9296-4872-8d58-5297ae31e830 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:25.938872 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d812a28a-9296-4872-8d58-5297ae31e830 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:25.943649 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0f6e0-048f-4cde-aa66-0c78fc97dcc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:25.944309 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0f6e0-048f-4cde-aa66-0c78fc97dcc7 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:25.944639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0f6e0-048f-4cde-aa66-0c78fc97dcc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:25.944919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e0f6e0-048f-4cde-aa66-0c78fc97dcc7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:25.945663 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0f6e0-048f-4cde-aa66-0c78fc97dcc7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:25.947604 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e0f6e0-048f-4cde-aa66-0c78fc97dcc7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:25.979020 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1297/2606] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:17:25 2022] POST /identity/v3/projects => generated 369 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:17:25.980225 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ca9ea-0748-4e8c-9052-2e56590e1790 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:25.990429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa4ca9ea-0748-4e8c-9052-2e56590e1790 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:26.049513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586f045-8f21-4ec3-9d3b-296b011913f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:26.050467 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586f045-8f21-4ec3-9d3b-296b011913f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:26.050750 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586f045-8f21-4ec3-9d3b-296b011913f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:26.051025 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d586f045-8f21-4ec3-9d3b-296b011913f8 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:26.051733 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586f045-8f21-4ec3-9d3b-296b011913f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:26.053654 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d586f045-8f21-4ec3-9d3b-296b011913f8 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:26.078221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1310/2607] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:17:25 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:17:26.079644 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0f6e0-048f-4cde-aa66-0c78fc97dcc7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:26.090534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e0f6e0-048f-4cde-aa66-0c78fc97dcc7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:26.095292 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfee14a-3ae0-43ef-88ff-843b0162e57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:26.096182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfee14a-3ae0-43ef-88ff-843b0162e57c admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:26.096480 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfee14a-3ae0-43ef-88ff-843b0162e57c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:26.096829 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfee14a-3ae0-43ef-88ff-843b0162e57c admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:26.097626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfee14a-3ae0-43ef-88ff-843b0162e57c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:26.099309 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfee14a-3ae0-43ef-88ff-843b0162e57c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:26.110613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1298/2608] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:26.114710 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586f045-8f21-4ec3-9d3b-296b011913f8 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:26.125829 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d586f045-8f21-4ec3-9d3b-296b011913f8 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:26.131575 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc68c573-dfde-468b-993d-10f165996e69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:26.132451 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68c573-dfde-468b-993d-10f165996e69 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:26.132744 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68c573-dfde-468b-993d-10f165996e69 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:26.133034 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc68c573-dfde-468b-993d-10f165996e69 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:26.133770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68c573-dfde-468b-993d-10f165996e69 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:26.135388 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc68c573-dfde-468b-993d-10f165996e69 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:26.145769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1311/2609] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:26 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:26.149909 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfee14a-3ae0-43ef-88ff-843b0162e57c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:26.159984 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfee14a-3ae0-43ef-88ff-843b0162e57c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:26.165848 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f8b41-35fb-4496-adda-bb8116b5ebac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:26.166790 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f8b41-35fb-4496-adda-bb8116b5ebac admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:26.167079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f8b41-35fb-4496-adda-bb8116b5ebac admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:26.167362 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b90f8b41-35fb-4496-adda-bb8116b5ebac admin admin] PATH_INFO: `/v3/projects/9a7d950a8bf049dcb5600fc64b3b6cc0/users/9ae9e78d25e94319801ce906327ada27/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:26.184952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f8b41-35fb-4496-adda-bb8116b5ebac admin admin] RBAC: Authorizing `identity:create_grant(project_id=9a7d950a8bf049dcb5600fc64b3b6cc0, user_id=9ae9e78d25e94319801ce906327ada27, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:26.187703 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b90f8b41-35fb-4496-adda-bb8116b5ebac admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:26.202287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1299/2610] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:26 2022] PUT /identity/v3/projects/9a7d950a8bf049dcb5600fc64b3b6cc0/users/9ae9e78d25e94319801ce906327ada27/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:26.207452 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82ad57-3dde-44d1-a56e-8e487cf86eb1 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:26.207745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82ad57-3dde-44d1-a56e-8e487cf86eb1 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:26.208028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b82ad57-3dde-44d1-a56e-8e487cf86eb1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:26.226759 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-2b82ad57-3dde-44d1-a56e-8e487cf86eb1 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:26.228069 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2b82ad57-3dde-44d1-a56e-8e487cf86eb1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:26.291109 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1312/2611] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:26.300874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f8b41-35fb-4496-adda-bb8116b5ebac admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:26.302711 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b90f8b41-35fb-4496-adda-bb8116b5ebac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:26.363140 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b90f8b41-35fb-4496-adda-bb8116b5ebac admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:26.366760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8320c-43c6-40bf-9d6b-2769f08bfe3c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:26.367420 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8320c-43c6-40bf-9d6b-2769f08bfe3c service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:26.367709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8320c-43c6-40bf-9d6b-2769f08bfe3c service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:26.367992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfa8320c-43c6-40bf-9d6b-2769f08bfe3c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:26.377364 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8320c-43c6-40bf-9d6b-2769f08bfe3c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:26.378871 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfa8320c-43c6-40bf-9d6b-2769f08bfe3c service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:26.389396 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1300/2612] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:27.106820 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea770b72-f36b-4253-bcaf-4cb6586ccc25 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:27.107354 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea770b72-f36b-4253-bcaf-4cb6586ccc25 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:27.108002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea770b72-f36b-4253-bcaf-4cb6586ccc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:27.143013 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-ea770b72-f36b-4253-bcaf-4cb6586ccc25 None None] MFA Rules not processed for user `9ae9e78d25e94319801ce906327ada27`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:27.144376 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ea770b72-f36b-4253-bcaf-4cb6586ccc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:27.218882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:27.229871 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1313/2613] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:27 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:17:27.238060 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8320c-43c6-40bf-9d6b-2769f08bfe3c service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:27.239872 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-bfa8320c-43c6-40bf-9d6b-2769f08bfe3c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:27.255491 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:27.316470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfa8320c-43c6-40bf-9d6b-2769f08bfe3c service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:27.320893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d63b4-eb88-4fd1-85c7-07417900b7ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:27.321572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d63b4-eb88-4fd1-85c7-07417900b7ba service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:27.321854 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d63b4-eb88-4fd1-85c7-07417900b7ba service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:27.322135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2d63b4-eb88-4fd1-85c7-07417900b7ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:27.331448 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d63b4-eb88-4fd1-85c7-07417900b7ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:27.332907 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2d63b4-eb88-4fd1-85c7-07417900b7ba service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:27.342909 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1301/2614] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:27 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:28.741837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be047e-d192-4a65-8693-b5d8efcf938e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:28.742623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be047e-d192-4a65-8693-b5d8efcf938e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:28.743195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63be047e-d192-4a65-8693-b5d8efcf938e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:28.764824 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-63be047e-d192-4a65-8693-b5d8efcf938e None None] MFA Rules not processed for user `9ae9e78d25e94319801ce906327ada27`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:28.766028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-63be047e-d192-4a65-8693-b5d8efcf938e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:28.773090 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1314/2615] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:28 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:29.384332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d63b4-eb88-4fd1-85c7-07417900b7ba service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:29.385857 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8c2d63b4-eb88-4fd1-85c7-07417900b7ba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:29.401025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2d63b4-eb88-4fd1-85c7-07417900b7ba service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:29.406267 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41d3e60b-8b49-480c-9040-597b150262dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:29.407179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3e60b-8b49-480c-9040-597b150262dc admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:29.407469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3e60b-8b49-480c-9040-597b150262dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:29.407757 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41d3e60b-8b49-480c-9040-597b150262dc admin admin] PATH_INFO: `/v3/users/04e86369906d4633b02a5ca1bcc24b35` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:29.423489 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3e60b-8b49-480c-9040-597b150262dc admin admin] RBAC: Authorizing `identity:delete_user(user_id=04e86369906d4633b02a5ca1bcc24b35)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:29.425684 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41d3e60b-8b49-480c-9040-597b150262dc admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:29.482706 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-41d3e60b-8b49-480c-9040-597b150262dc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04e86369906d4633b02a5ca1bcc24b35'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:29.483346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-41d3e60b-8b49-480c-9040-597b150262dc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04e86369906d4633b02a5ca1bcc24b35'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:29.494410 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-41d3e60b-8b49-480c-9040-597b150262dc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04e86369906d4633b02a5ca1bcc24b35'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:29.495065 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-41d3e60b-8b49-480c-9040-597b150262dc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04e86369906d4633b02a5ca1bcc24b35'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:29.509035 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-41d3e60b-8b49-480c-9040-597b150262dc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04e86369906d4633b02a5ca1bcc24b35'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:29.521399 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1302/2616] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:17:29 2022] DELETE /identity/v3/users/04e86369906d4633b02a5ca1bcc24b35 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:29.733045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be047e-d192-4a65-8693-b5d8efcf938e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:29.747465 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63be047e-d192-4a65-8693-b5d8efcf938e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:29.817207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18046134-b85c-43b0-8130-637330e1f994 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:29.818653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18046134-b85c-43b0-8130-637330e1f994 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:29.818653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18046134-b85c-43b0-8130-637330e1f994 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:29.818653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18046134-b85c-43b0-8130-637330e1f994 admin admin] PATH_INFO: `/v3/projects/da749564df7941929c6cb9159479d637` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:29.832135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18046134-b85c-43b0-8130-637330e1f994 admin admin] RBAC: Authorizing `identity:delete_project(project_id=da749564df7941929c6cb9159479d637)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:29.834066 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18046134-b85c-43b0-8130-637330e1f994 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:29.900401 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-18046134-b85c-43b0-8130-637330e1f994 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'da749564df7941929c6cb9159479d637', 'request_id': 'req-18046134-b85c-43b0-8130-637330e1f994'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:29.910546 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-18046134-b85c-43b0-8130-637330e1f994 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'da749564df7941929c6cb9159479d637', 'request_id': 'req-18046134-b85c-43b0-8130-637330e1f994'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:29.918161 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-18046134-b85c-43b0-8130-637330e1f994 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'da749564df7941929c6cb9159479d637', 'request_id': 'req-18046134-b85c-43b0-8130-637330e1f994'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:29.933160 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-18046134-b85c-43b0-8130-637330e1f994 admin admin] The token cache is being invalidate because project da749564df7941929c6cb9159479d637 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:17:29.933405 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-18046134-b85c-43b0-8130-637330e1f994 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:29.934438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-18046134-b85c-43b0-8130-637330e1f994 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:29.936425 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1315/2617] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:17:29 2022] DELETE /identity/v3/projects/da749564df7941929c6cb9159479d637 => generated 0 bytes in 204 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:29.941527 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6fed3-d67f-48af-aec9-41b58f113a2b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:29.941853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6fed3-d67f-48af-aec9-41b58f113a2b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:29.942341 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fd6fed3-d67f-48af-aec9-41b58f113a2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:29.968616 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-5fd6fed3-d67f-48af-aec9-41b58f113a2b None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:29.970127 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5fd6fed3-d67f-48af-aec9-41b58f113a2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:30.049615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1303/2618] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:17:30.058236 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999b8bb-3c6d-44a6-b6a4-b6e1e4bc50bb None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:30.058571 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999b8bb-3c6d-44a6-b6a4-b6e1e4bc50bb None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:30.058857 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9999b8bb-3c6d-44a6-b6a4-b6e1e4bc50bb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:30.079230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9999b8bb-3c6d-44a6-b6a4-b6e1e4bc50bb None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:30.080518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9999b8bb-3c6d-44a6-b6a4-b6e1e4bc50bb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:30.088734 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1316/2619] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:17:30.092976 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6fed3-d67f-48af-aec9-41b58f113a2b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:30.104404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fd6fed3-d67f-48af-aec9-41b58f113a2b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:30.110736 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee11f9a-64d6-4404-8079-962fd149ca6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:30.111462 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee11f9a-64d6-4404-8079-962fd149ca6e admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:30.111703 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee11f9a-64d6-4404-8079-962fd149ca6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:30.111981 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee11f9a-64d6-4404-8079-962fd149ca6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:30.112726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee11f9a-64d6-4404-8079-962fd149ca6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:17:30.113041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee11f9a-64d6-4404-8079-962fd149ca6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:30.114679 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee11f9a-64d6-4404-8079-962fd149ca6e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:30.122730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:30.133864 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1304/2620] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:17:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 11 14:17:30.135008 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999b8bb-3c6d-44a6-b6a4-b6e1e4bc50bb None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:30.144377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9999b8bb-3c6d-44a6-b6a4-b6e1e4bc50bb None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:30.149323 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda29b5-69c8-498e-b707-672adcd17696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:30.149986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda29b5-69c8-498e-b707-672adcd17696 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:30.150272 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda29b5-69c8-498e-b707-672adcd17696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:30.150614 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fda29b5-69c8-498e-b707-672adcd17696 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:30.151315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda29b5-69c8-498e-b707-672adcd17696 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:30.153255 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fda29b5-69c8-498e-b707-672adcd17696 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:30.183993 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1317/2621] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:17:30 2022] POST /identity/v3/projects => generated 373 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:17:30.187867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee11f9a-64d6-4404-8079-962fd149ca6e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:30.198904 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee11f9a-64d6-4404-8079-962fd149ca6e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:30.259716 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d78a01-0067-4613-b268-543820a52558 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:30.260548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d78a01-0067-4613-b268-543820a52558 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:30.260745 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d78a01-0067-4613-b268-543820a52558 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:30.261055 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d78a01-0067-4613-b268-543820a52558 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:30.261801 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d78a01-0067-4613-b268-543820a52558 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:30.263865 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d78a01-0067-4613-b268-543820a52558 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:30.284937 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1305/2622] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:17:30 2022] POST /identity/v3/users => generated 287 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:30.287941 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda29b5-69c8-498e-b707-672adcd17696 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:30.299221 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fda29b5-69c8-498e-b707-672adcd17696 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:30.304573 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c923ee-a7c9-40a2-a134-220c3b8d9f45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:30.305240 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c923ee-a7c9-40a2-a134-220c3b8d9f45 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:30.305526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c923ee-a7c9-40a2-a134-220c3b8d9f45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:30.305805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1c923ee-a7c9-40a2-a134-220c3b8d9f45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:30.306609 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c923ee-a7c9-40a2-a134-220c3b8d9f45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:30.308211 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1c923ee-a7c9-40a2-a134-220c3b8d9f45 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:30.318796 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1318/2623] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:30.322861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d78a01-0067-4613-b268-543820a52558 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:30.333644 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d78a01-0067-4613-b268-543820a52558 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:30.339015 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2c4519f-9935-4fdd-b9a8-25135884a620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:30.339672 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4519f-9935-4fdd-b9a8-25135884a620 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:30.339956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4519f-9935-4fdd-b9a8-25135884a620 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:30.340258 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2c4519f-9935-4fdd-b9a8-25135884a620 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:30.341017 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4519f-9935-4fdd-b9a8-25135884a620 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:30.342730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2c4519f-9935-4fdd-b9a8-25135884a620 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:30.353806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1306/2624] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:30 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:30.358192 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c923ee-a7c9-40a2-a134-220c3b8d9f45 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:30.368560 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1c923ee-a7c9-40a2-a134-220c3b8d9f45 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:30.373134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0483ff1-e71d-4b2d-8652-e983c6d2ee5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:30.374030 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0483ff1-e71d-4b2d-8652-e983c6d2ee5b admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:30.374329 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0483ff1-e71d-4b2d-8652-e983c6d2ee5b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:30.374615 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0483ff1-e71d-4b2d-8652-e983c6d2ee5b admin admin] PATH_INFO: `/v3/projects/e2bb24cc72064066b16a7321bbe0c1d4/users/2d80b3c2b07b4684a81e11d7e528fdc3/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:30.389881 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0483ff1-e71d-4b2d-8652-e983c6d2ee5b admin admin] RBAC: Authorizing `identity:create_grant(project_id=e2bb24cc72064066b16a7321bbe0c1d4, user_id=2d80b3c2b07b4684a81e11d7e528fdc3, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:30.392583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0483ff1-e71d-4b2d-8652-e983c6d2ee5b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:30.405887 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1319/2625] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:30 2022] PUT /identity/v3/projects/e2bb24cc72064066b16a7321bbe0c1d4/users/2d80b3c2b07b4684a81e11d7e528fdc3/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:30.416730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc4e97-76a4-4286-a13c-e9873bca7fef None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:30.417258 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc4e97-76a4-4286-a13c-e9873bca7fef None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:30.417258 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3cc4e97-76a4-4286-a13c-e9873bca7fef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:30.443969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-e3cc4e97-76a4-4286-a13c-e9873bca7fef None None] MFA Rules not processed for user `2d80b3c2b07b4684a81e11d7e528fdc3`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:30.445797 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-e3cc4e97-76a4-4286-a13c-e9873bca7fef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:30.509538 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1307/2626] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:17:30.520940 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0483ff1-e71d-4b2d-8652-e983c6d2ee5b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:30.522768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a0483ff1-e71d-4b2d-8652-e983c6d2ee5b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:30.536240 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:30.603332 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0483ff1-e71d-4b2d-8652-e983c6d2ee5b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:30.609589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07c3cd77-5872-4d40-95b1-def146ac0c36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:30.610283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3cd77-5872-4d40-95b1-def146ac0c36 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:30.610597 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3cd77-5872-4d40-95b1-def146ac0c36 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:30.610877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07c3cd77-5872-4d40-95b1-def146ac0c36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:30.620038 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3cd77-5872-4d40-95b1-def146ac0c36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:30.621628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07c3cd77-5872-4d40-95b1-def146ac0c36 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:30.631865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1320/2627] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:30.893361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc4e97-76a4-4286-a13c-e9873bca7fef None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:30.964310 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3cc4e97-76a4-4286-a13c-e9873bca7fef None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:30.968876 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0346fb83-9bb2-433c-b67e-df1f1bb89671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:30.969779 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0346fb83-9bb2-433c-b67e-df1f1bb89671 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:30.970084 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0346fb83-9bb2-433c-b67e-df1f1bb89671 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:30.970492 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0346fb83-9bb2-433c-b67e-df1f1bb89671 admin admin] PATH_INFO: `/v3/users/2d80b3c2b07b4684a81e11d7e528fdc3` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:30.971620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0346fb83-9bb2-433c-b67e-df1f1bb89671 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2d80b3c2b07b4684a81e11d7e528fdc3)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:30.973531 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0346fb83-9bb2-433c-b67e-df1f1bb89671 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:30.980977 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:31.026979 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0346fb83-9bb2-433c-b67e-df1f1bb89671 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2d80b3c2b07b4684a81e11d7e528fdc3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:31.027639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0346fb83-9bb2-433c-b67e-df1f1bb89671 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2d80b3c2b07b4684a81e11d7e528fdc3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:31.036451 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0346fb83-9bb2-433c-b67e-df1f1bb89671 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2d80b3c2b07b4684a81e11d7e528fdc3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:31.037429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0346fb83-9bb2-433c-b67e-df1f1bb89671 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2d80b3c2b07b4684a81e11d7e528fdc3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:31.054027 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0346fb83-9bb2-433c-b67e-df1f1bb89671 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2d80b3c2b07b4684a81e11d7e528fdc3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:31.067003 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1308/2628] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:17:30 2022] DELETE /identity/v3/users/2d80b3c2b07b4684a81e11d7e528fdc3 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:31.071652 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e5cd5-7033-4259-9de8-25975f72436c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:31.072157 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e5cd5-7033-4259-9de8-25975f72436c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:31.072277 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331e5cd5-7033-4259-9de8-25975f72436c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:31.095966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-331e5cd5-7033-4259-9de8-25975f72436c None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:31.097502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-331e5cd5-7033-4259-9de8-25975f72436c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:31.166952 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1321/2629] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:17:31.177287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0346fb83-9bb2-433c-b67e-df1f1bb89671 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:31.178860 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0346fb83-9bb2-433c-b67e-df1f1bb89671 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:31.248333 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0346fb83-9bb2-433c-b67e-df1f1bb89671 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:31.252273 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8dd5152-7312-47df-8033-d6a3beab1d0d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:31.253048 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd5152-7312-47df-8033-d6a3beab1d0d service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:31.253342 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd5152-7312-47df-8033-d6a3beab1d0d service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:31.253621 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8dd5152-7312-47df-8033-d6a3beab1d0d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:31.263525 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd5152-7312-47df-8033-d6a3beab1d0d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:31.265013 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8dd5152-7312-47df-8033-d6a3beab1d0d service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:31.275741 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1309/2630] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:31.555583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e5cd5-7033-4259-9de8-25975f72436c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:31.557232 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-331e5cd5-7033-4259-9de8-25975f72436c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:31.574618 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331e5cd5-7033-4259-9de8-25975f72436c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:31.582166 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-567ab4be-2d0e-47e3-93cf-588576be7ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:31.583447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ab4be-2d0e-47e3-93cf-588576be7ac9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:31.583910 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ab4be-2d0e-47e3-93cf-588576be7ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:31.584471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-567ab4be-2d0e-47e3-93cf-588576be7ac9 admin admin] PATH_INFO: `/v3/projects/e2bb24cc72064066b16a7321bbe0c1d4` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:31.586358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567ab4be-2d0e-47e3-93cf-588576be7ac9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e2bb24cc72064066b16a7321bbe0c1d4)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:31.588395 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-567ab4be-2d0e-47e3-93cf-588576be7ac9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:31.637415 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-567ab4be-2d0e-47e3-93cf-588576be7ac9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e2bb24cc72064066b16a7321bbe0c1d4', 'request_id': 'req-567ab4be-2d0e-47e3-93cf-588576be7ac9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:31.646375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-567ab4be-2d0e-47e3-93cf-588576be7ac9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e2bb24cc72064066b16a7321bbe0c1d4', 'request_id': 'req-567ab4be-2d0e-47e3-93cf-588576be7ac9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:31.653733 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-567ab4be-2d0e-47e3-93cf-588576be7ac9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e2bb24cc72064066b16a7321bbe0c1d4', 'request_id': 'req-567ab4be-2d0e-47e3-93cf-588576be7ac9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:31.665719 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-567ab4be-2d0e-47e3-93cf-588576be7ac9 admin admin] The token cache is being invalidate because project e2bb24cc72064066b16a7321bbe0c1d4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:17:31.666006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-567ab4be-2d0e-47e3-93cf-588576be7ac9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:31.666704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-567ab4be-2d0e-47e3-93cf-588576be7ac9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:31.668324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1322/2631] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:17:31 2022] DELETE /identity/v3/projects/e2bb24cc72064066b16a7321bbe0c1d4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:31.673601 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bbc16-d603-437e-b96b-af82398ee775 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:31.674110 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bbc16-d603-437e-b96b-af82398ee775 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:31.674273 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94bbc16-d603-437e-b96b-af82398ee775 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:31.693848 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-d94bbc16-d603-437e-b96b-af82398ee775 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:31.695711 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d94bbc16-d603-437e-b96b-af82398ee775 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:31.756042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1310/2632] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:31.764579 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff20e7-a121-4d50-8c63-f63ef89856a3 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:31.764868 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff20e7-a121-4d50-8c63-f63ef89856a3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:31.765170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbff20e7-a121-4d50-8c63-f63ef89856a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:31.786588 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-bbff20e7-a121-4d50-8c63-f63ef89856a3 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:31.786734 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-bbff20e7-a121-4d50-8c63-f63ef89856a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:31.795794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1323/2633] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:31.800567 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bbc16-d603-437e-b96b-af82398ee775 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:31.811753 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94bbc16-d603-437e-b96b-af82398ee775 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:31.819095 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60c046-77e3-48ef-958b-9e91108b2afb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:31.819095 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60c046-77e3-48ef-958b-9e91108b2afb admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:31.820155 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60c046-77e3-48ef-958b-9e91108b2afb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:31.820325 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc60c046-77e3-48ef-958b-9e91108b2afb admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:31.821048 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60c046-77e3-48ef-958b-9e91108b2afb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:17:31.821369 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60c046-77e3-48ef-958b-9e91108b2afb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:31.823149 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc60c046-77e3-48ef-958b-9e91108b2afb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:31.835659 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1311/2634] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:17:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:17:31.840038 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff20e7-a121-4d50-8c63-f63ef89856a3 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:31.849664 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbff20e7-a121-4d50-8c63-f63ef89856a3 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:31.854596 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05bf95-2e86-4231-94b5-b10777ee542e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:31.855274 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05bf95-2e86-4231-94b5-b10777ee542e admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:31.855560 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05bf95-2e86-4231-94b5-b10777ee542e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:31.855838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc05bf95-2e86-4231-94b5-b10777ee542e admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:31.856723 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05bf95-2e86-4231-94b5-b10777ee542e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:31.858696 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc05bf95-2e86-4231-94b5-b10777ee542e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:31.871928 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:31.895274 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1324/2635] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:17:31 2022] POST /identity/v3/projects => generated 363 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:31.899615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60c046-77e3-48ef-958b-9e91108b2afb admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:31.912499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc60c046-77e3-48ef-958b-9e91108b2afb admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:31.929887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:31.987919 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f4639-ddde-4be3-a7b7-65816013661b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:31.988643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f4639-ddde-4be3-a7b7-65816013661b admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:31.988934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f4639-ddde-4be3-a7b7-65816013661b admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:31.989216 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35f4639-ddde-4be3-a7b7-65816013661b admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:31.989929 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f4639-ddde-4be3-a7b7-65816013661b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:31.992582 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e35f4639-ddde-4be3-a7b7-65816013661b admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:32.014567 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1312/2636] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:17:31 2022] POST /identity/v3/users => generated 282 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:32.018900 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05bf95-2e86-4231-94b5-b10777ee542e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:32.029765 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc05bf95-2e86-4231-94b5-b10777ee542e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:32.034723 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836553e3-403c-459b-9a70-ae54b6b614bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:32.035405 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836553e3-403c-459b-9a70-ae54b6b614bb admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:32.035691 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836553e3-403c-459b-9a70-ae54b6b614bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:32.035971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-836553e3-403c-459b-9a70-ae54b6b614bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:32.036714 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836553e3-403c-459b-9a70-ae54b6b614bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:32.038440 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-836553e3-403c-459b-9a70-ae54b6b614bb admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:32.051767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1325/2637] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:32.056065 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f4639-ddde-4be3-a7b7-65816013661b admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:32.066758 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35f4639-ddde-4be3-a7b7-65816013661b admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:32.072748 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d2d88e1-c86c-4cd7-b113-b08f7ab59847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:32.073447 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d88e1-c86c-4cd7-b113-b08f7ab59847 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:32.073736 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d88e1-c86c-4cd7-b113-b08f7ab59847 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:32.074020 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d2d88e1-c86c-4cd7-b113-b08f7ab59847 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:32.074826 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d88e1-c86c-4cd7-b113-b08f7ab59847 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:32.076455 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d2d88e1-c86c-4cd7-b113-b08f7ab59847 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:32.087665 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1313/2638] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:32.092099 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836553e3-403c-459b-9a70-ae54b6b614bb admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:32.104082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-836553e3-403c-459b-9a70-ae54b6b614bb admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:32.109087 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fbadb-84a8-4c13-824b-0dae49d350a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:32.110157 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fbadb-84a8-4c13-824b-0dae49d350a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:32.110483 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fbadb-84a8-4c13-824b-0dae49d350a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:32.110762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-985fbadb-84a8-4c13-824b-0dae49d350a3 admin admin] PATH_INFO: `/v3/projects/2de47fbbac2d4ff58cc75d3dc0020f6d/users/8d8ca192027444dd874f528f3b6a1843/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:32.127985 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fbadb-84a8-4c13-824b-0dae49d350a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2de47fbbac2d4ff58cc75d3dc0020f6d, user_id=8d8ca192027444dd874f528f3b6a1843, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:32.130786 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-985fbadb-84a8-4c13-824b-0dae49d350a3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:32.145667 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1326/2639] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:32 2022] PUT /identity/v3/projects/2de47fbbac2d4ff58cc75d3dc0020f6d/users/8d8ca192027444dd874f528f3b6a1843/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:32.157533 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922b543-fd74-4e5b-8183-9ec737a32a28 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:32.157843 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922b543-fd74-4e5b-8183-9ec737a32a28 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:32.158113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b922b543-fd74-4e5b-8183-9ec737a32a28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:32.188433 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b922b543-fd74-4e5b-8183-9ec737a32a28 None None] MFA Rules not processed for user `8d8ca192027444dd874f528f3b6a1843`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:32.189675 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b922b543-fd74-4e5b-8183-9ec737a32a28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:32.262452 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1314/2640] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:32 2022] POST /identity/v3/auth/tokens => generated 3230 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:32.267342 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fbadb-84a8-4c13-824b-0dae49d350a3 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:32.338456 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-985fbadb-84a8-4c13-824b-0dae49d350a3 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:32.344459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94d5da-c99f-4dcb-86de-27638843741b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:32.345257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94d5da-c99f-4dcb-86de-27638843741b admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:32.345549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94d5da-c99f-4dcb-86de-27638843741b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:32.345867 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af94d5da-c99f-4dcb-86de-27638843741b admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:32.346698 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94d5da-c99f-4dcb-86de-27638843741b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:32.348679 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af94d5da-c99f-4dcb-86de-27638843741b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:32.381139 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1327/2641] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:17:32 2022] POST /identity/v3/projects => generated 365 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:32.385137 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922b543-fd74-4e5b-8183-9ec737a32a28 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:32.398636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b922b543-fd74-4e5b-8183-9ec737a32a28 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:32.483392 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1aab87-43d4-472e-bdc7-62771e62d8f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:32.484084 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1aab87-43d4-472e-bdc7-62771e62d8f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:32.484997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1aab87-43d4-472e-bdc7-62771e62d8f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:32.484997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d1aab87-43d4-472e-bdc7-62771e62d8f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:32.485350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1aab87-43d4-472e-bdc7-62771e62d8f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:32.487316 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d1aab87-43d4-472e-bdc7-62771e62d8f7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:32.513798 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1315/2642] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:17:32 2022] POST /identity/v3/users => generated 289 bytes in 126 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:32.515142 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94d5da-c99f-4dcb-86de-27638843741b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:32.526132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af94d5da-c99f-4dcb-86de-27638843741b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:32.531223 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f0ee5-90ef-4833-b878-c9019c7592d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:32.531901 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f0ee5-90ef-4833-b878-c9019c7592d9 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:32.532187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f0ee5-90ef-4833-b878-c9019c7592d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:32.532465 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca2f0ee5-90ef-4833-b878-c9019c7592d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:32.533207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f0ee5-90ef-4833-b878-c9019c7592d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:32.534931 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca2f0ee5-90ef-4833-b878-c9019c7592d9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:32.547615 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1328/2643] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:17:32.552055 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1aab87-43d4-472e-bdc7-62771e62d8f7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:32.563234 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d1aab87-43d4-472e-bdc7-62771e62d8f7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:32.568527 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a00ad4-680f-4743-82b1-f43c4668fa18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:32.570185 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a00ad4-680f-4743-82b1-f43c4668fa18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:32.570520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a00ad4-680f-4743-82b1-f43c4668fa18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:32.570920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7a00ad4-680f-4743-82b1-f43c4668fa18 admin admin] PATH_INFO: `/v3/projects/a7b947f182b54cedb9273bb005168d34/users/1e8d832117b54e8da614bc066f39818e/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:32.583023 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:32.588002 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a00ad4-680f-4743-82b1-f43c4668fa18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7b947f182b54cedb9273bb005168d34, user_id=1e8d832117b54e8da614bc066f39818e, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:32.590537 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7a00ad4-680f-4743-82b1-f43c4668fa18 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:32.606521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1316/2644] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:32 2022] PUT /identity/v3/projects/a7b947f182b54cedb9273bb005168d34/users/1e8d832117b54e8da614bc066f39818e/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:32.610578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f0ee5-90ef-4833-b878-c9019c7592d9 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:32.623355 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca2f0ee5-90ef-4833-b878-c9019c7592d9 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:32.682789 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b6db3-7488-41f5-a23d-7e12afe9d0ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:32.683466 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b6db3-7488-41f5-a23d-7e12afe9d0ea admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:32.683756 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b6db3-7488-41f5-a23d-7e12afe9d0ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:32.684039 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b6db3-7488-41f5-a23d-7e12afe9d0ea admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:32.684777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b6db3-7488-41f5-a23d-7e12afe9d0ea admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:32.686439 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b6db3-7488-41f5-a23d-7e12afe9d0ea admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:32.697207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1329/2645] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:32 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:32.701464 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a00ad4-680f-4743-82b1-f43c4668fa18 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:32.712379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7a00ad4-680f-4743-82b1-f43c4668fa18 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:32.718633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e77bb8-bc37-499d-817d-87cb49659ca6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:32.719536 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e77bb8-bc37-499d-817d-87cb49659ca6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:32.719826 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e77bb8-bc37-499d-817d-87cb49659ca6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:32.720116 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e77bb8-bc37-499d-817d-87cb49659ca6 admin admin] PATH_INFO: `/v3/projects/a7b947f182b54cedb9273bb005168d34/users/1e8d832117b54e8da614bc066f39818e/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:32.722151 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e77bb8-bc37-499d-817d-87cb49659ca6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a7b947f182b54cedb9273bb005168d34, user_id=1e8d832117b54e8da614bc066f39818e, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:32.724835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e77bb8-bc37-499d-817d-87cb49659ca6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:32.736525 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1317/2646] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:32 2022] PUT /identity/v3/projects/a7b947f182b54cedb9273bb005168d34/users/1e8d832117b54e8da614bc066f39818e/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:32.747386 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08717e45-aff0-4380-8cc6-85e654633e86 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:32.747682 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08717e45-aff0-4380-8cc6-85e654633e86 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:32.747965 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08717e45-aff0-4380-8cc6-85e654633e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:32.775185 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-08717e45-aff0-4380-8cc6-85e654633e86 None None] MFA Rules not processed for user `1e8d832117b54e8da614bc066f39818e`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:32.776430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-08717e45-aff0-4380-8cc6-85e654633e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:32.789202 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:32.848319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1330/2647] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:32 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:32.860999 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e77bb8-bc37-499d-817d-87cb49659ca6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:32.862958 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-47e77bb8-bc37-499d-817d-87cb49659ca6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:32.922104 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e77bb8-bc37-499d-817d-87cb49659ca6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:32.926088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598912d6-5421-4925-8acd-349feff712c3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:32.926779 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598912d6-5421-4925-8acd-349feff712c3 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:32.927065 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598912d6-5421-4925-8acd-349feff712c3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:32.927344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598912d6-5421-4925-8acd-349feff712c3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:32.936247 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598912d6-5421-4925-8acd-349feff712c3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:32.937667 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598912d6-5421-4925-8acd-349feff712c3 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:32.947507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1318/2648] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:32 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:17:33.024701 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08717e45-aff0-4380-8cc6-85e654633e86 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:33.034884 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08717e45-aff0-4380-8cc6-85e654633e86 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:33.038722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39cbd53f-40c7-4d78-b7bb-da17761ee247 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:33.039377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbd53f-40c7-4d78-b7bb-da17761ee247 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:33.039691 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbd53f-40c7-4d78-b7bb-da17761ee247 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:33.039972 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39cbd53f-40c7-4d78-b7bb-da17761ee247 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:33.103917 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbd53f-40c7-4d78-b7bb-da17761ee247 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:33.105365 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39cbd53f-40c7-4d78-b7bb-da17761ee247 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:33.115180 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1331/2649] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:33 2022] GET /identity/v3/auth/tokens => generated 3230 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:35.822780 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598912d6-5421-4925-8acd-349feff712c3 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:35.886482 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598912d6-5421-4925-8acd-349feff712c3 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:35.891207 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a143391-8152-4036-8ecc-111219f66dbb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:35.892086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a143391-8152-4036-8ecc-111219f66dbb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:35.892374 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a143391-8152-4036-8ecc-111219f66dbb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:35.892653 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a143391-8152-4036-8ecc-111219f66dbb admin admin] PATH_INFO: `/v3/users/8d8ca192027444dd874f528f3b6a1843` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:35.893850 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a143391-8152-4036-8ecc-111219f66dbb admin admin] RBAC: Authorizing `identity:delete_user(user_id=8d8ca192027444dd874f528f3b6a1843)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:35.895816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a143391-8152-4036-8ecc-111219f66dbb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:35.944417 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5a143391-8152-4036-8ecc-111219f66dbb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8d8ca192027444dd874f528f3b6a1843'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:35.945087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5a143391-8152-4036-8ecc-111219f66dbb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8d8ca192027444dd874f528f3b6a1843'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:35.953129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5a143391-8152-4036-8ecc-111219f66dbb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8d8ca192027444dd874f528f3b6a1843'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:35.953816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5a143391-8152-4036-8ecc-111219f66dbb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8d8ca192027444dd874f528f3b6a1843'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:35.966662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5a143391-8152-4036-8ecc-111219f66dbb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8d8ca192027444dd874f528f3b6a1843'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:35.974341 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:35.981316 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1319/2650] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:17:35 2022] DELETE /identity/v3/users/8d8ca192027444dd874f528f3b6a1843 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:35.986925 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e370fcd-c1d5-4411-a3d5-b047c9248c11 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:35.987663 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e370fcd-c1d5-4411-a3d5-b047c9248c11 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:35.987663 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e370fcd-c1d5-4411-a3d5-b047c9248c11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:36.007036 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1e370fcd-c1d5-4411-a3d5-b047c9248c11 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:36.008211 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1e370fcd-c1d5-4411-a3d5-b047c9248c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:36.071249 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1332/2651] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:36.081298 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a143391-8152-4036-8ecc-111219f66dbb admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:36.082766 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5a143391-8152-4036-8ecc-111219f66dbb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:36.141816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a143391-8152-4036-8ecc-111219f66dbb admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:36.145541 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd68cf1-0e2c-42f4-9ad5-45a11e585085 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:36.146204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd68cf1-0e2c-42f4-9ad5-45a11e585085 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:36.146505 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd68cf1-0e2c-42f4-9ad5-45a11e585085 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:36.146789 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd68cf1-0e2c-42f4-9ad5-45a11e585085 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:36.155936 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd68cf1-0e2c-42f4-9ad5-45a11e585085 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:36.157353 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd68cf1-0e2c-42f4-9ad5-45a11e585085 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:36.167507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1320/2652] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:36.429837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e370fcd-c1d5-4411-a3d5-b047c9248c11 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:36.431410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1e370fcd-c1d5-4411-a3d5-b047c9248c11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:36.443686 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e370fcd-c1d5-4411-a3d5-b047c9248c11 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:36.448236 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0a68a-1c34-44a3-a82e-e3f145b6838c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:36.449146 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0a68a-1c34-44a3-a82e-e3f145b6838c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:36.449434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0a68a-1c34-44a3-a82e-e3f145b6838c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:36.449720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ff0a68a-1c34-44a3-a82e-e3f145b6838c admin admin] PATH_INFO: `/v3/projects/2de47fbbac2d4ff58cc75d3dc0020f6d` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:36.451028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0a68a-1c34-44a3-a82e-e3f145b6838c admin admin] RBAC: Authorizing `identity:delete_project(project_id=2de47fbbac2d4ff58cc75d3dc0020f6d)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:36.452952 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ff0a68a-1c34-44a3-a82e-e3f145b6838c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:36.514039 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:36.517411 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6ff0a68a-1c34-44a3-a82e-e3f145b6838c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2de47fbbac2d4ff58cc75d3dc0020f6d', 'request_id': 'req-6ff0a68a-1c34-44a3-a82e-e3f145b6838c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:36.528250 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6ff0a68a-1c34-44a3-a82e-e3f145b6838c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2de47fbbac2d4ff58cc75d3dc0020f6d', 'request_id': 'req-6ff0a68a-1c34-44a3-a82e-e3f145b6838c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:36.535796 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6ff0a68a-1c34-44a3-a82e-e3f145b6838c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2de47fbbac2d4ff58cc75d3dc0020f6d', 'request_id': 'req-6ff0a68a-1c34-44a3-a82e-e3f145b6838c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:36.549206 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6ff0a68a-1c34-44a3-a82e-e3f145b6838c admin admin] The token cache is being invalidate because project 2de47fbbac2d4ff58cc75d3dc0020f6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:17:36.549600 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6ff0a68a-1c34-44a3-a82e-e3f145b6838c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:36.550653 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6ff0a68a-1c34-44a3-a82e-e3f145b6838c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:36.552671 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1333/2653] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:17:36 2022] DELETE /identity/v3/projects/2de47fbbac2d4ff58cc75d3dc0020f6d => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:36.557503 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd68cf1-0e2c-42f4-9ad5-45a11e585085 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:36.568273 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd68cf1-0e2c-42f4-9ad5-45a11e585085 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:36.646003 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fa6528-0587-4849-84cd-63f81da78e43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:36.646945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa6528-0587-4849-84cd-63f81da78e43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:36.647218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa6528-0587-4849-84cd-63f81da78e43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:36.648103 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fa6528-0587-4849-84cd-63f81da78e43 admin admin] PATH_INFO: `/v3/users/1e8d832117b54e8da614bc066f39818e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:36.649587 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa6528-0587-4849-84cd-63f81da78e43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1e8d832117b54e8da614bc066f39818e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:36.651634 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fa6528-0587-4849-84cd-63f81da78e43 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:36.701915 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e2fa6528-0587-4849-84cd-63f81da78e43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1e8d832117b54e8da614bc066f39818e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:36.702619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e2fa6528-0587-4849-84cd-63f81da78e43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1e8d832117b54e8da614bc066f39818e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:36.711900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e2fa6528-0587-4849-84cd-63f81da78e43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1e8d832117b54e8da614bc066f39818e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:36.712558 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e2fa6528-0587-4849-84cd-63f81da78e43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1e8d832117b54e8da614bc066f39818e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:36.727034 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-e2fa6528-0587-4849-84cd-63f81da78e43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1e8d832117b54e8da614bc066f39818e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:36.740862 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1321/2654] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:17:36 2022] DELETE /identity/v3/users/1e8d832117b54e8da614bc066f39818e => generated 0 bytes in 184 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:17:37.021373 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0a68a-1c34-44a3-a82e-e3f145b6838c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:37.023512 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-6ff0a68a-1c34-44a3-a82e-e3f145b6838c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:37.037223 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ff0a68a-1c34-44a3-a82e-e3f145b6838c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:37.097768 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d92a51-f6b7-4ba0-a065-28556406a252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:37.098781 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d92a51-f6b7-4ba0-a065-28556406a252 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:37.099066 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d92a51-f6b7-4ba0-a065-28556406a252 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:37.099342 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d92a51-f6b7-4ba0-a065-28556406a252 admin admin] PATH_INFO: `/v3/projects/a7b947f182b54cedb9273bb005168d34` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:37.101158 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d92a51-f6b7-4ba0-a065-28556406a252 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a7b947f182b54cedb9273bb005168d34)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:37.104132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d92a51-f6b7-4ba0-a065-28556406a252 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:37.167394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-27d92a51-f6b7-4ba0-a065-28556406a252 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a7b947f182b54cedb9273bb005168d34', 'request_id': 'req-27d92a51-f6b7-4ba0-a065-28556406a252'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:37.176603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-27d92a51-f6b7-4ba0-a065-28556406a252 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a7b947f182b54cedb9273bb005168d34', 'request_id': 'req-27d92a51-f6b7-4ba0-a065-28556406a252'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:37.183147 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-27d92a51-f6b7-4ba0-a065-28556406a252 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a7b947f182b54cedb9273bb005168d34', 'request_id': 'req-27d92a51-f6b7-4ba0-a065-28556406a252'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:37.195041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-27d92a51-f6b7-4ba0-a065-28556406a252 admin admin] The token cache is being invalidate because project a7b947f182b54cedb9273bb005168d34 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:17:37.195352 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-27d92a51-f6b7-4ba0-a065-28556406a252 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:37.196022 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-27d92a51-f6b7-4ba0-a065-28556406a252 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:37.197602 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1334/2655] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:17:37 2022] DELETE /identity/v3/projects/a7b947f182b54cedb9273bb005168d34 => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:37.203300 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f998d-bc65-4462-bfe3-20102bdee894 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:37.203971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f998d-bc65-4462-bfe3-20102bdee894 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:37.204352 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943f998d-bc65-4462-bfe3-20102bdee894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:37.224684 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-943f998d-bc65-4462-bfe3-20102bdee894 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:37.226518 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-943f998d-bc65-4462-bfe3-20102bdee894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:37.241250 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:37.301142 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1322/2656] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:37.309064 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36dbd8-84cc-4205-8f74-9313393f3638 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:37.309356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36dbd8-84cc-4205-8f74-9313393f3638 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:37.309637 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be36dbd8-84cc-4205-8f74-9313393f3638 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:37.328430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-be36dbd8-84cc-4205-8f74-9313393f3638 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:37.329640 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-be36dbd8-84cc-4205-8f74-9313393f3638 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:37.338386 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1335/2657] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:37.342519 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f998d-bc65-4462-bfe3-20102bdee894 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:37.352171 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943f998d-bc65-4462-bfe3-20102bdee894 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:37.356633 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7e034-dbb6-4736-8548-6c88c84b1ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:37.357308 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7e034-dbb6-4736-8548-6c88c84b1ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:37.357589 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7e034-dbb6-4736-8548-6c88c84b1ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:37.357863 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b7e034-dbb6-4736-8548-6c88c84b1ea7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:37.358626 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7e034-dbb6-4736-8548-6c88c84b1ea7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:17:37.358934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7e034-dbb6-4736-8548-6c88c84b1ea7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:37.360508 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b7e034-dbb6-4736-8548-6c88c84b1ea7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:37.372702 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1323/2658] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:17:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:17:37.377108 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36dbd8-84cc-4205-8f74-9313393f3638 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:37.386816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be36dbd8-84cc-4205-8f74-9313393f3638 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:37.391400 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9c606-f772-4c37-a21f-36aaeab4c368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:37.392045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9c606-f772-4c37-a21f-36aaeab4c368 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:37.392333 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9c606-f772-4c37-a21f-36aaeab4c368 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:37.392608 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c9c606-f772-4c37-a21f-36aaeab4c368 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:37.393303 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9c606-f772-4c37-a21f-36aaeab4c368 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:37.395226 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c9c606-f772-4c37-a21f-36aaeab4c368 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:37.420466 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1336/2659] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:17:37 2022] POST /identity/v3/projects => generated 367 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:37.424418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7e034-dbb6-4736-8548-6c88c84b1ea7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:37.434961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b7e034-dbb6-4736-8548-6c88c84b1ea7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:37.489786 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fad88-81e5-4cdc-8b00-63270d869f49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:37.490492 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fad88-81e5-4cdc-8b00-63270d869f49 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:37.490778 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fad88-81e5-4cdc-8b00-63270d869f49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:37.491080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e7fad88-81e5-4cdc-8b00-63270d869f49 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:37.491770 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7fad88-81e5-4cdc-8b00-63270d869f49 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:37.493656 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e7fad88-81e5-4cdc-8b00-63270d869f49 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:37.513554 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1324/2660] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:17:37 2022] POST /identity/v3/users => generated 284 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:17:37.517663 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9c606-f772-4c37-a21f-36aaeab4c368 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:37.528400 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c9c606-f772-4c37-a21f-36aaeab4c368 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:37.532994 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9061c59-5093-45cf-87e5-9e75465e106e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:37.533641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9061c59-5093-45cf-87e5-9e75465e106e admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:37.533922 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9061c59-5093-45cf-87e5-9e75465e106e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:37.534210 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9061c59-5093-45cf-87e5-9e75465e106e admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:37.534959 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9061c59-5093-45cf-87e5-9e75465e106e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:37.536563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9061c59-5093-45cf-87e5-9e75465e106e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:37.547146 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1337/2661] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:37.551433 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fad88-81e5-4cdc-8b00-63270d869f49 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:37.562008 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e7fad88-81e5-4cdc-8b00-63270d869f49 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:37.566826 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9c86aef-7b15-4fe6-80c9-2b8c7ad0ce18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:37.567506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c86aef-7b15-4fe6-80c9-2b8c7ad0ce18 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:37.567796 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c86aef-7b15-4fe6-80c9-2b8c7ad0ce18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:37.568075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9c86aef-7b15-4fe6-80c9-2b8c7ad0ce18 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:37.568976 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c86aef-7b15-4fe6-80c9-2b8c7ad0ce18 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:37.571340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9c86aef-7b15-4fe6-80c9-2b8c7ad0ce18 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:37.583194 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1325/2662] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:37.588360 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9061c59-5093-45cf-87e5-9e75465e106e admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:37.598717 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9061c59-5093-45cf-87e5-9e75465e106e admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:37.603468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cde73-5a46-4ec3-a945-adca9d11ea94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:37.604348 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cde73-5a46-4ec3-a945-adca9d11ea94 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:37.604634 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cde73-5a46-4ec3-a945-adca9d11ea94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:37.604914 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f91cde73-5a46-4ec3-a945-adca9d11ea94 admin admin] PATH_INFO: `/v3/projects/d937a8015a8f488c8657c12f30d8175f/users/b07c3aa30d514496884b712f73ee5823/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:37.621274 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91cde73-5a46-4ec3-a945-adca9d11ea94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d937a8015a8f488c8657c12f30d8175f, user_id=b07c3aa30d514496884b712f73ee5823, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:37.624453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f91cde73-5a46-4ec3-a945-adca9d11ea94 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:37.634740 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:37.640163 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1338/2663] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:37 2022] PUT /identity/v3/projects/d937a8015a8f488c8657c12f30d8175f/users/b07c3aa30d514496884b712f73ee5823/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:37.652299 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad481ff-3580-4aed-8a9d-ee4af82fc2eb None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:37.652580 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad481ff-3580-4aed-8a9d-ee4af82fc2eb None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:37.652867 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad481ff-3580-4aed-8a9d-ee4af82fc2eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:37.681743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0ad481ff-3580-4aed-8a9d-ee4af82fc2eb None None] MFA Rules not processed for user `b07c3aa30d514496884b712f73ee5823`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:37.683043 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0ad481ff-3580-4aed-8a9d-ee4af82fc2eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:37.751428 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1326/2664] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:37 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:37.756086 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cde73-5a46-4ec3-a945-adca9d11ea94 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:37.821339 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f91cde73-5a46-4ec3-a945-adca9d11ea94 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:37.826132 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65dc3a3-b14c-4e63-a684-888fc65420c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:37.826835 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dc3a3-b14c-4e63-a684-888fc65420c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:37.827131 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dc3a3-b14c-4e63-a684-888fc65420c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:37.827411 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d65dc3a3-b14c-4e63-a684-888fc65420c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:37.828097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65dc3a3-b14c-4e63-a684-888fc65420c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:37.830033 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d65dc3a3-b14c-4e63-a684-888fc65420c6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:37.855866 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1339/2665] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:17:37 2022] POST /identity/v3/projects => generated 365 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:17:37.859715 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad481ff-3580-4aed-8a9d-ee4af82fc2eb None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:37.870745 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ad481ff-3580-4aed-8a9d-ee4af82fc2eb None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:37.888674 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:37.932625 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d40002-f4c0-42d6-85e9-73bcaea7a2c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:37.933216 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d40002-f4c0-42d6-85e9-73bcaea7a2c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:37.933526 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d40002-f4c0-42d6-85e9-73bcaea7a2c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:37.933801 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2d40002-f4c0-42d6-85e9-73bcaea7a2c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:37.934561 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d40002-f4c0-42d6-85e9-73bcaea7a2c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:37.936464 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2d40002-f4c0-42d6-85e9-73bcaea7a2c6 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:37.958324 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1327/2666] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:17:37 2022] POST /identity/v3/users => generated 289 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:37.962323 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65dc3a3-b14c-4e63-a684-888fc65420c6 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:37.972438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d65dc3a3-b14c-4e63-a684-888fc65420c6 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:37.976989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bb6af-6016-4c4c-a295-14f74f4aa016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:37.977643 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bb6af-6016-4c4c-a295-14f74f4aa016 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:37.977926 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bb6af-6016-4c4c-a295-14f74f4aa016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:37.978204 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4bb6af-6016-4c4c-a295-14f74f4aa016 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:37.978965 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bb6af-6016-4c4c-a295-14f74f4aa016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:37.980553 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f4bb6af-6016-4c4c-a295-14f74f4aa016 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:37.991501 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1340/2667] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:37 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:37.995968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d40002-f4c0-42d6-85e9-73bcaea7a2c6 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:38.005846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2d40002-f4c0-42d6-85e9-73bcaea7a2c6 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:38.011178 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e69f99-42ac-4802-bf66-9abfcda7bc26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:38.012117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e69f99-42ac-4802-bf66-9abfcda7bc26 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:38.012408 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e69f99-42ac-4802-bf66-9abfcda7bc26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:38.012683 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e69f99-42ac-4802-bf66-9abfcda7bc26 admin admin] PATH_INFO: `/v3/projects/9bf7b09acb6641aeb8f10ec594819913/users/a9401a7b2470491a8252843a26bc23f3/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:38.027204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e69f99-42ac-4802-bf66-9abfcda7bc26 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf7b09acb6641aeb8f10ec594819913, user_id=a9401a7b2470491a8252843a26bc23f3, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:38.029839 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76e69f99-42ac-4802-bf66-9abfcda7bc26 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:38.042661 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1328/2668] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:37 2022] PUT /identity/v3/projects/9bf7b09acb6641aeb8f10ec594819913/users/a9401a7b2470491a8252843a26bc23f3/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:38.046762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bb6af-6016-4c4c-a295-14f74f4aa016 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:38.059306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4bb6af-6016-4c4c-a295-14f74f4aa016 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:38.118151 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffe1872-adfc-4423-9193-8801f6ab9f4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:38.118852 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe1872-adfc-4423-9193-8801f6ab9f4a admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:38.119141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe1872-adfc-4423-9193-8801f6ab9f4a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:38.119417 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffe1872-adfc-4423-9193-8801f6ab9f4a admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:38.120144 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe1872-adfc-4423-9193-8801f6ab9f4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:38.121741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffe1872-adfc-4423-9193-8801f6ab9f4a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:38.131591 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1341/2669] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:38 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:38.135866 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e69f99-42ac-4802-bf66-9abfcda7bc26 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:38.146021 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e69f99-42ac-4802-bf66-9abfcda7bc26 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:38.150654 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4400b44-6bb7-4079-8faa-229c8277bed2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:38.151537 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4400b44-6bb7-4079-8faa-229c8277bed2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:38.151820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4400b44-6bb7-4079-8faa-229c8277bed2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:38.152097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4400b44-6bb7-4079-8faa-229c8277bed2 admin admin] PATH_INFO: `/v3/projects/9bf7b09acb6641aeb8f10ec594819913/users/a9401a7b2470491a8252843a26bc23f3/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:38.154056 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4400b44-6bb7-4079-8faa-229c8277bed2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=9bf7b09acb6641aeb8f10ec594819913, user_id=a9401a7b2470491a8252843a26bc23f3, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:38.156706 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4400b44-6bb7-4079-8faa-229c8277bed2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:38.167588 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1329/2670] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:38 2022] PUT /identity/v3/projects/9bf7b09acb6641aeb8f10ec594819913/users/a9401a7b2470491a8252843a26bc23f3/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:38.177880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8809ac-9d2b-4c9e-b7fa-8c7998b31705 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:38.178179 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8809ac-9d2b-4c9e-b7fa-8c7998b31705 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:38.178550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8809ac-9d2b-4c9e-b7fa-8c7998b31705 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:38.205364 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-cb8809ac-9d2b-4c9e-b7fa-8c7998b31705 None None] MFA Rules not processed for user `a9401a7b2470491a8252843a26bc23f3`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:38.206538 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-cb8809ac-9d2b-4c9e-b7fa-8c7998b31705 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:38.263954 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:38.274065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1342/2671] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:38 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:38.285712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4400b44-6bb7-4079-8faa-229c8277bed2 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:38.287566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a4400b44-6bb7-4079-8faa-229c8277bed2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:38.346046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4400b44-6bb7-4079-8faa-229c8277bed2 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:38.349798 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54698ad-f436-458c-a1e8-9c0adfda70a5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:38.350526 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54698ad-f436-458c-a1e8-9c0adfda70a5 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:38.350815 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54698ad-f436-458c-a1e8-9c0adfda70a5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:38.351094 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54698ad-f436-458c-a1e8-9c0adfda70a5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:38.360464 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54698ad-f436-458c-a1e8-9c0adfda70a5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:38.361895 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d54698ad-f436-458c-a1e8-9c0adfda70a5 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:38.371857 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1330/2672] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:38 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:38.453299 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8809ac-9d2b-4c9e-b7fa-8c7998b31705 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:38.464001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8809ac-9d2b-4c9e-b7fa-8c7998b31705 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:38.467794 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec546b15-63fe-4e11-a230-b10d27fc255c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:38.468505 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec546b15-63fe-4e11-a230-b10d27fc255c service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:38.468798 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec546b15-63fe-4e11-a230-b10d27fc255c service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:38.469087 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec546b15-63fe-4e11-a230-b10d27fc255c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:38.529088 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec546b15-63fe-4e11-a230-b10d27fc255c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:38.530614 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec546b15-63fe-4e11-a230-b10d27fc255c service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:38.540120 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1343/2673] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:38 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:43.271060 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54698ad-f436-458c-a1e8-9c0adfda70a5 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:43.337651 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54698ad-f436-458c-a1e8-9c0adfda70a5 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:43.342264 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff164d-e047-493b-b25b-a76498e07726 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:43.343172 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff164d-e047-493b-b25b-a76498e07726 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:43.343476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff164d-e047-493b-b25b-a76498e07726 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:43.343753 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ff164d-e047-493b-b25b-a76498e07726 admin admin] PATH_INFO: `/v3/users/b07c3aa30d514496884b712f73ee5823` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:43.344947 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff164d-e047-493b-b25b-a76498e07726 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b07c3aa30d514496884b712f73ee5823)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:43.346975 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ff164d-e047-493b-b25b-a76498e07726 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:43.360682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=94,keystone:INSERT=5,keystone:DELETE=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:43.395531 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d0ff164d-e047-493b-b25b-a76498e07726 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b07c3aa30d514496884b712f73ee5823'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:43.396206 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d0ff164d-e047-493b-b25b-a76498e07726 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b07c3aa30d514496884b712f73ee5823'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:43.403504 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d0ff164d-e047-493b-b25b-a76498e07726 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b07c3aa30d514496884b712f73ee5823'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:43.404244 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d0ff164d-e047-493b-b25b-a76498e07726 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b07c3aa30d514496884b712f73ee5823'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:43.417033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-d0ff164d-e047-493b-b25b-a76498e07726 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b07c3aa30d514496884b712f73ee5823'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:43.429392 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1331/2674] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:17:43 2022] DELETE /identity/v3/users/b07c3aa30d514496884b712f73ee5823 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:43.434001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19790d8-4a8f-4416-8eb5-e1d7dc65d358 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:43.434514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19790d8-4a8f-4416-8eb5-e1d7dc65d358 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:43.434625 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c19790d8-4a8f-4416-8eb5-e1d7dc65d358 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:43.453386 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c19790d8-4a8f-4416-8eb5-e1d7dc65d358 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:43.454642 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c19790d8-4a8f-4416-8eb5-e1d7dc65d358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:43.513153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1344/2675] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:43.522784 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff164d-e047-493b-b25b-a76498e07726 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:43.525143 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-d0ff164d-e047-493b-b25b-a76498e07726 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:43.582895 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ff164d-e047-493b-b25b-a76498e07726 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:43.587185 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98ffa5-861e-46de-9127-4d2c737dd278 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:43.587874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98ffa5-861e-46de-9127-4d2c737dd278 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:43.588127 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98ffa5-861e-46de-9127-4d2c737dd278 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:43.588398 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca98ffa5-861e-46de-9127-4d2c737dd278 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:43.597830 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca98ffa5-861e-46de-9127-4d2c737dd278 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:43.599271 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca98ffa5-861e-46de-9127-4d2c737dd278 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:43.609614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1332/2676] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:43.875858 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19790d8-4a8f-4416-8eb5-e1d7dc65d358 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:43.877411 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c19790d8-4a8f-4416-8eb5-e1d7dc65d358 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:43.897686 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c19790d8-4a8f-4416-8eb5-e1d7dc65d358 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:43.902633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5c8dd-8f88-425c-9abd-978aa1fab217 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:43.903454 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5c8dd-8f88-425c-9abd-978aa1fab217 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:43.903746 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5c8dd-8f88-425c-9abd-978aa1fab217 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:43.904030 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc5c8dd-8f88-425c-9abd-978aa1fab217 admin admin] PATH_INFO: `/v3/projects/d937a8015a8f488c8657c12f30d8175f` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:43.905256 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc5c8dd-8f88-425c-9abd-978aa1fab217 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d937a8015a8f488c8657c12f30d8175f)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:43.907227 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc5c8dd-8f88-425c-9abd-978aa1fab217 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:43.955189 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4fc5c8dd-8f88-425c-9abd-978aa1fab217 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd937a8015a8f488c8657c12f30d8175f', 'request_id': 'req-4fc5c8dd-8f88-425c-9abd-978aa1fab217'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:43.968639 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4fc5c8dd-8f88-425c-9abd-978aa1fab217 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd937a8015a8f488c8657c12f30d8175f', 'request_id': 'req-4fc5c8dd-8f88-425c-9abd-978aa1fab217'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:43.975195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4fc5c8dd-8f88-425c-9abd-978aa1fab217 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd937a8015a8f488c8657c12f30d8175f', 'request_id': 'req-4fc5c8dd-8f88-425c-9abd-978aa1fab217'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:43.986745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4fc5c8dd-8f88-425c-9abd-978aa1fab217 admin admin] The token cache is being invalidate because project d937a8015a8f488c8657c12f30d8175f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:17:43.987036 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4fc5c8dd-8f88-425c-9abd-978aa1fab217 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:43.987698 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-4fc5c8dd-8f88-425c-9abd-978aa1fab217 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:43.989261 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1345/2677] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:17:43 2022] DELETE /identity/v3/projects/d937a8015a8f488c8657c12f30d8175f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:43.993534 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98ffa5-861e-46de-9127-4d2c737dd278 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:44.003663 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca98ffa5-861e-46de-9127-4d2c737dd278 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:44.063564 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b1cd2-351f-445b-9d68-9f8024d3ffcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:44.064379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b1cd2-351f-445b-9d68-9f8024d3ffcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:44.064666 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b1cd2-351f-445b-9d68-9f8024d3ffcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:44.064941 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b1b1cd2-351f-445b-9d68-9f8024d3ffcb admin admin] PATH_INFO: `/v3/users/a9401a7b2470491a8252843a26bc23f3` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:44.066161 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b1cd2-351f-445b-9d68-9f8024d3ffcb admin admin] RBAC: Authorizing `identity:delete_user(user_id=a9401a7b2470491a8252843a26bc23f3)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:44.068104 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b1b1cd2-351f-445b-9d68-9f8024d3ffcb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:44.107871 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:44.120267 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8b1b1cd2-351f-445b-9d68-9f8024d3ffcb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a9401a7b2470491a8252843a26bc23f3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:44.121047 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8b1b1cd2-351f-445b-9d68-9f8024d3ffcb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a9401a7b2470491a8252843a26bc23f3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:44.128556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8b1b1cd2-351f-445b-9d68-9f8024d3ffcb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a9401a7b2470491a8252843a26bc23f3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:44.129191 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8b1b1cd2-351f-445b-9d68-9f8024d3ffcb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a9401a7b2470491a8252843a26bc23f3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:44.142182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8b1b1cd2-351f-445b-9d68-9f8024d3ffcb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a9401a7b2470491a8252843a26bc23f3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:44.156786 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1333/2678] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:17:43 2022] DELETE /identity/v3/users/a9401a7b2470491a8252843a26bc23f3 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:44.462175 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5c8dd-8f88-425c-9abd-978aa1fab217 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:44.463864 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4fc5c8dd-8f88-425c-9abd-978aa1fab217 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:44.490460 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:44.511847 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b1cd2-351f-445b-9d68-9f8024d3ffcb admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:44.513615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8b1b1cd2-351f-445b-9d68-9f8024d3ffcb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:44.572511 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc5c8dd-8f88-425c-9abd-978aa1fab217 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:44.593763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b1b1cd2-351f-445b-9d68-9f8024d3ffcb admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:44.598874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e69108-e91f-4472-9d2b-12e2b4231245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:44.599750 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e69108-e91f-4472-9d2b-12e2b4231245 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:44.600397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e69108-e91f-4472-9d2b-12e2b4231245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:44.600397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e69108-e91f-4472-9d2b-12e2b4231245 admin admin] PATH_INFO: `/v3/projects/9bf7b09acb6641aeb8f10ec594819913` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:44.602172 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e69108-e91f-4472-9d2b-12e2b4231245 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9bf7b09acb6641aeb8f10ec594819913)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:44.604158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2e69108-e91f-4472-9d2b-12e2b4231245 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:44.643882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fc986c8-e032-4ad1-b562-e3f509821d3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:44.644549 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc986c8-e032-4ad1-b562-e3f509821d3d service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:44.644838 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc986c8-e032-4ad1-b562-e3f509821d3d service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:44.645120 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fc986c8-e032-4ad1-b562-e3f509821d3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:44.647230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4fc986c8-e032-4ad1-b562-e3f509821d3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:44.656819 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c2e69108-e91f-4472-9d2b-12e2b4231245 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9bf7b09acb6641aeb8f10ec594819913', 'request_id': 'req-c2e69108-e91f-4472-9d2b-12e2b4231245'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:44.659743 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc986c8-e032-4ad1-b562-e3f509821d3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:44.661333 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fc986c8-e032-4ad1-b562-e3f509821d3d service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:44.663399 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c2e69108-e91f-4472-9d2b-12e2b4231245 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9bf7b09acb6641aeb8f10ec594819913', 'request_id': 'req-c2e69108-e91f-4472-9d2b-12e2b4231245'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:44.671909 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c2e69108-e91f-4472-9d2b-12e2b4231245 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9bf7b09acb6641aeb8f10ec594819913', 'request_id': 'req-c2e69108-e91f-4472-9d2b-12e2b4231245'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:44.674041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1346/2679] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:44 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:44.688620 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c2e69108-e91f-4472-9d2b-12e2b4231245 admin admin] The token cache is being invalidate because project 9bf7b09acb6641aeb8f10ec594819913 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:17:44.688955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c2e69108-e91f-4472-9d2b-12e2b4231245 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:44.689558 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c2e69108-e91f-4472-9d2b-12e2b4231245 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:44.691158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1334/2680] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:17:44 2022] DELETE /identity/v3/projects/9bf7b09acb6641aeb8f10ec594819913 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:17:44.697065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4141c1-1b99-4e9e-8568-bbd14fdfee46 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:44.697584 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4141c1-1b99-4e9e-8568-bbd14fdfee46 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:44.697872 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4141c1-1b99-4e9e-8568-bbd14fdfee46 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:44.718148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-cc4141c1-1b99-4e9e-8568-bbd14fdfee46 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:44.719371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-cc4141c1-1b99-4e9e-8568-bbd14fdfee46 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:44.777515 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1347/2681] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:44.785376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708d45e-7c85-43b8-b2e9-6652e09b3e75 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:44.785674 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708d45e-7c85-43b8-b2e9-6652e09b3e75 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:44.785952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1708d45e-7c85-43b8-b2e9-6652e09b3e75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:44.805407 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1708d45e-7c85-43b8-b2e9-6652e09b3e75 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:44.806638 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1708d45e-7c85-43b8-b2e9-6652e09b3e75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:44.814507 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1335/2682] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:44.818601 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4141c1-1b99-4e9e-8568-bbd14fdfee46 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:44.828224 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4141c1-1b99-4e9e-8568-bbd14fdfee46 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:44.832783 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d089f-08bd-4e58-bdab-a37be141a2b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:44.833434 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d089f-08bd-4e58-bdab-a37be141a2b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:44.833719 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d089f-08bd-4e58-bdab-a37be141a2b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:44.833997 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d089f-08bd-4e58-bdab-a37be141a2b3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:44.834757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d089f-08bd-4e58-bdab-a37be141a2b3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:17:44.835068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d089f-08bd-4e58-bdab-a37be141a2b3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:44.837150 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d089f-08bd-4e58-bdab-a37be141a2b3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:44.848770 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1348/2683] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:17:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:17:44.853641 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708d45e-7c85-43b8-b2e9-6652e09b3e75 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:44.862992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1708d45e-7c85-43b8-b2e9-6652e09b3e75 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:44.867862 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7a00f-aafd-4e8f-a63b-c16097f952f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:44.868528 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7a00f-aafd-4e8f-a63b-c16097f952f8 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:44.868816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7a00f-aafd-4e8f-a63b-c16097f952f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:44.869094 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7a00f-aafd-4e8f-a63b-c16097f952f8 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:44.869804 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7a00f-aafd-4e8f-a63b-c16097f952f8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:44.871759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7a00f-aafd-4e8f-a63b-c16097f952f8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:44.900769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1336/2684] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:17:44 2022] POST /identity/v3/projects => generated 361 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:17:44.904722 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d089f-08bd-4e58-bdab-a37be141a2b3 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:44.918138 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d089f-08bd-4e58-bdab-a37be141a2b3 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:44.948919 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:44.984024 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31fc4a-2263-43cc-b2a7-f623a6c5b119 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:44.984835 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31fc4a-2263-43cc-b2a7-f623a6c5b119 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:44.985126 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31fc4a-2263-43cc-b2a7-f623a6c5b119 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:44.985404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb31fc4a-2263-43cc-b2a7-f623a6c5b119 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:44.986110 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31fc4a-2263-43cc-b2a7-f623a6c5b119 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:44.988072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb31fc4a-2263-43cc-b2a7-f623a6c5b119 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:45.014839 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1349/2685] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:17:44 2022] POST /identity/v3/users => generated 281 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:45.019306 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7a00f-aafd-4e8f-a63b-c16097f952f8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:45.031398 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7a00f-aafd-4e8f-a63b-c16097f952f8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:45.036105 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483bb6b-5bd6-4595-9d85-a0501308d549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:45.036779 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483bb6b-5bd6-4595-9d85-a0501308d549 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:45.037070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483bb6b-5bd6-4595-9d85-a0501308d549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:45.037350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8483bb6b-5bd6-4595-9d85-a0501308d549 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:45.038088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8483bb6b-5bd6-4595-9d85-a0501308d549 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:45.040302 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8483bb6b-5bd6-4595-9d85-a0501308d549 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:45.054237 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1337/2686] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:45.058345 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31fc4a-2263-43cc-b2a7-f623a6c5b119 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:45.068955 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb31fc4a-2263-43cc-b2a7-f623a6c5b119 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:45.073709 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d09c5b-9206-44b1-83c0-fda5f2fe2e8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:45.074416 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d09c5b-9206-44b1-83c0-fda5f2fe2e8a admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:45.074703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d09c5b-9206-44b1-83c0-fda5f2fe2e8a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:45.074976 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d09c5b-9206-44b1-83c0-fda5f2fe2e8a admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:45.075734 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d09c5b-9206-44b1-83c0-fda5f2fe2e8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:45.077331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d09c5b-9206-44b1-83c0-fda5f2fe2e8a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:45.088282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1350/2687] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:45 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:45.092527 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483bb6b-5bd6-4595-9d85-a0501308d549 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:45.097833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=85,keystone:DELETE=12,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:45.104520 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8483bb6b-5bd6-4595-9d85-a0501308d549 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:45.111628 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0534758-4f8f-4c71-b2c2-52a89446b482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:45.112539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0534758-4f8f-4c71-b2c2-52a89446b482 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:45.113014 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0534758-4f8f-4c71-b2c2-52a89446b482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:45.113559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0534758-4f8f-4c71-b2c2-52a89446b482 admin admin] PATH_INFO: `/v3/projects/461c622098ec4dfcbac8c1744e94a9c4/users/4a9ebc7803804e56a0891668db79bd6d/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:45.128478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0534758-4f8f-4c71-b2c2-52a89446b482 admin admin] RBAC: Authorizing `identity:create_grant(project_id=461c622098ec4dfcbac8c1744e94a9c4, user_id=4a9ebc7803804e56a0891668db79bd6d, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:45.131139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0534758-4f8f-4c71-b2c2-52a89446b482 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:45.143627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1338/2688] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:45 2022] PUT /identity/v3/projects/461c622098ec4dfcbac8c1744e94a9c4/users/4a9ebc7803804e56a0891668db79bd6d/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:45.155707 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d08bb-6bd3-4055-9831-bebe26eac12a None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:45.155995 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d08bb-6bd3-4055-9831-bebe26eac12a None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:45.156278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-071d08bb-6bd3-4055-9831-bebe26eac12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:45.187289 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-071d08bb-6bd3-4055-9831-bebe26eac12a None None] MFA Rules not processed for user `4a9ebc7803804e56a0891668db79bd6d`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:45.188553 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-071d08bb-6bd3-4055-9831-bebe26eac12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:45.256698 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1351/2689] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:45 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:45.288590 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0534758-4f8f-4c71-b2c2-52a89446b482 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:45.290397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a0534758-4f8f-4c71-b2c2-52a89446b482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:45.353452 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0534758-4f8f-4c71-b2c2-52a89446b482 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:45.357308 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80e23c1b-0847-4ea9-ae72-4b993cd04507 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:45.358040 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e23c1b-0847-4ea9-ae72-4b993cd04507 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:45.358340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e23c1b-0847-4ea9-ae72-4b993cd04507 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:45.358544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80e23c1b-0847-4ea9-ae72-4b993cd04507 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:45.367908 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e23c1b-0847-4ea9-ae72-4b993cd04507 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:45.369334 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80e23c1b-0847-4ea9-ae72-4b993cd04507 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:45.378687 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1339/2690] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:45 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:45.393797 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d08bb-6bd3-4055-9831-bebe26eac12a None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:45.458198 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-071d08bb-6bd3-4055-9831-bebe26eac12a None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:45.463471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a158844-eda3-46cc-844a-117095bb7566 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:45.464350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a158844-eda3-46cc-844a-117095bb7566 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:45.464634 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a158844-eda3-46cc-844a-117095bb7566 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:45.464914 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a158844-eda3-46cc-844a-117095bb7566 admin admin] PATH_INFO: `/v3/users/4a9ebc7803804e56a0891668db79bd6d` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:45.466124 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a158844-eda3-46cc-844a-117095bb7566 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4a9ebc7803804e56a0891668db79bd6d)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:45.468087 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a158844-eda3-46cc-844a-117095bb7566 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:45.511111 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:45.518881 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6a158844-eda3-46cc-844a-117095bb7566 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4a9ebc7803804e56a0891668db79bd6d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:45.534330 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6a158844-eda3-46cc-844a-117095bb7566 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4a9ebc7803804e56a0891668db79bd6d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:45.548953 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6a158844-eda3-46cc-844a-117095bb7566 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4a9ebc7803804e56a0891668db79bd6d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:45.549607 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6a158844-eda3-46cc-844a-117095bb7566 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4a9ebc7803804e56a0891668db79bd6d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:45.552095 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5819900-7061-4312-a189-f92a95364c81 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:45.552821 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5819900-7061-4312-a189-f92a95364c81 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:45.553145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5819900-7061-4312-a189-f92a95364c81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:45.558038 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-6a158844-eda3-46cc-844a-117095bb7566 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4a9ebc7803804e56a0891668db79bd6d'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:45.564606 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1352/2691] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:17:45 2022] DELETE /identity/v3/users/4a9ebc7803804e56a0891668db79bd6d => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:45.569235 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e93bc-b61c-42c3-8414-1265e2586765 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:45.569510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e93bc-b61c-42c3-8414-1265e2586765 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:45.569911 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-579e93bc-b61c-42c3-8414-1265e2586765 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:45.572531 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-f5819900-7061-4312-a189-f92a95364c81 None None] MFA Rules not processed for user `9ae9e78d25e94319801ce906327ada27`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:45.575972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f5819900-7061-4312-a189-f92a95364c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:45.590017 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-579e93bc-b61c-42c3-8414-1265e2586765 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:45.591278 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-579e93bc-b61c-42c3-8414-1265e2586765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:45.638087 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1340/2692] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:45 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:45.656900 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1353/2693] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:17:45.667414 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5819900-7061-4312-a189-f92a95364c81 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:45.668868 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f5819900-7061-4312-a189-f92a95364c81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:45.728570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5819900-7061-4312-a189-f92a95364c81 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:45.733419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a0714-d889-4b08-9ec8-47e8b615ef51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:45.733776 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a0714-d889-4b08-9ec8-47e8b615ef51 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:45.734189 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a0714-d889-4b08-9ec8-47e8b615ef51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:45.734481 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a82a0714-d889-4b08-9ec8-47e8b615ef51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:45.745091 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a0714-d889-4b08-9ec8-47e8b615ef51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:45.746564 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a82a0714-d889-4b08-9ec8-47e8b615ef51 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:45.756528 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1341/2694] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:17:45.883447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e93bc-b61c-42c3-8414-1265e2586765 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:45.885349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-579e93bc-b61c-42c3-8414-1265e2586765 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:45.897789 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-579e93bc-b61c-42c3-8414-1265e2586765 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:45.907956 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b1ea836-efbd-49a8-8b46-cdb7732a33b4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:45.908641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ea836-efbd-49a8-8b46-cdb7732a33b4 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:45.908920 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ea836-efbd-49a8-8b46-cdb7732a33b4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:45.909781 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b1ea836-efbd-49a8-8b46-cdb7732a33b4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:45.911093 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-4b1ea836-efbd-49a8-8b46-cdb7732a33b4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:45.922986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ea836-efbd-49a8-8b46-cdb7732a33b4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:45.924507 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b1ea836-efbd-49a8-8b46-cdb7732a33b4 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:45.933403 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1354/2695] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:45 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:17:46.129128 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a0714-d889-4b08-9ec8-47e8b615ef51 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:46.130639 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a82a0714-d889-4b08-9ec8-47e8b615ef51 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:46.146277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a82a0714-d889-4b08-9ec8-47e8b615ef51 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:46.153113 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ca9bde-8c95-4b2d-8d48-7574497917dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:46.153991 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9bde-8c95-4b2d-8d48-7574497917dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:46.154277 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9bde-8c95-4b2d-8d48-7574497917dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:46.155052 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ca9bde-8c95-4b2d-8d48-7574497917dd admin admin] PATH_INFO: `/v3/projects/461c622098ec4dfcbac8c1744e94a9c4` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:46.155917 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca9bde-8c95-4b2d-8d48-7574497917dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=461c622098ec4dfcbac8c1744e94a9c4)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:46.157859 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ca9bde-8c95-4b2d-8d48-7574497917dd admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:46.185545 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:46.212747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-95ca9bde-8c95-4b2d-8d48-7574497917dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '461c622098ec4dfcbac8c1744e94a9c4', 'request_id': 'req-95ca9bde-8c95-4b2d-8d48-7574497917dd'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:46.218378 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-95ca9bde-8c95-4b2d-8d48-7574497917dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '461c622098ec4dfcbac8c1744e94a9c4', 'request_id': 'req-95ca9bde-8c95-4b2d-8d48-7574497917dd'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:46.227648 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-95ca9bde-8c95-4b2d-8d48-7574497917dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '461c622098ec4dfcbac8c1744e94a9c4', 'request_id': 'req-95ca9bde-8c95-4b2d-8d48-7574497917dd'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:46.238900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-95ca9bde-8c95-4b2d-8d48-7574497917dd admin admin] The token cache is being invalidate because project 461c622098ec4dfcbac8c1744e94a9c4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:17:46.239196 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-95ca9bde-8c95-4b2d-8d48-7574497917dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:46.239825 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-95ca9bde-8c95-4b2d-8d48-7574497917dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:46.241346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1342/2696] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:17:46 2022] DELETE /identity/v3/projects/461c622098ec4dfcbac8c1744e94a9c4 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:46.246262 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608075e8-7bf7-426d-887b-533c51df1adb None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:46.246592 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608075e8-7bf7-426d-887b-533c51df1adb None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:46.246879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-608075e8-7bf7-426d-887b-533c51df1adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:46.265879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-608075e8-7bf7-426d-887b-533c51df1adb None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:46.267148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-608075e8-7bf7-426d-887b-533c51df1adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:46.329994 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1355/2697] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:46.338595 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb5425-bd83-4f26-9a5e-ae51552fc8f5 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:46.338971 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb5425-bd83-4f26-9a5e-ae51552fc8f5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:46.339313 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bb5425-bd83-4f26-9a5e-ae51552fc8f5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:46.341567 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608075e8-7bf7-426d-887b-533c51df1adb None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:46.343447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-608075e8-7bf7-426d-887b-533c51df1adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:46.360167 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b0bb5425-bd83-4f26-9a5e-ae51552fc8f5 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:46.361514 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b0bb5425-bd83-4f26-9a5e-ae51552fc8f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:46.371390 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1343/2698] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:46.375782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb5425-bd83-4f26-9a5e-ae51552fc8f5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:46.388007 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bb5425-bd83-4f26-9a5e-ae51552fc8f5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:46.394707 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742bb92-dfa1-47de-be25-d7d443fe9b26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:46.395374 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742bb92-dfa1-47de-be25-d7d443fe9b26 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:46.395661 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742bb92-dfa1-47de-be25-d7d443fe9b26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:46.395938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0742bb92-dfa1-47de-be25-d7d443fe9b26 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:46.396671 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742bb92-dfa1-47de-be25-d7d443fe9b26 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:17:46.396980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742bb92-dfa1-47de-be25-d7d443fe9b26 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:46.398711 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0742bb92-dfa1-47de-be25-d7d443fe9b26 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:46.400783 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:DELETE=5,keystone:SELECT=94,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:46.411738 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1344/2699] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:17:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:17:46.416122 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742bb92-dfa1-47de-be25-d7d443fe9b26 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:46.427790 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0742bb92-dfa1-47de-be25-d7d443fe9b26 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:46.428790 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-608075e8-7bf7-426d-887b-533c51df1adb None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:46.433109 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8099b-354a-466e-8d97-dda181b6c755 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:46.433768 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8099b-354a-466e-8d97-dda181b6c755 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:46.434229 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8099b-354a-466e-8d97-dda181b6c755 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:46.434571 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb8099b-354a-466e-8d97-dda181b6c755 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:46.434700 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37acc86-765f-4bdd-bee0-42e23c09a61c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee29b7f1cea64d6781e1df5c8021363f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:46.435273 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8099b-354a-466e-8d97-dda181b6c755 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:46.435403 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37acc86-765f-4bdd-bee0-42e23c09a61c service placement] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:46.435678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37acc86-765f-4bdd-bee0-42e23c09a61c service placement] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:46.435962 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a37acc86-765f-4bdd-bee0-42e23c09a61c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:46.437204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb8099b-354a-466e-8d97-dda181b6c755 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:46.437758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a37acc86-765f-4bdd-bee0-42e23c09a61c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:46.450678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37acc86-765f-4bdd-bee0-42e23c09a61c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:46.452283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a37acc86-765f-4bdd-bee0-42e23c09a61c service placement] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:46.462535 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1356/2700] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:46.464986 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1345/2701] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:17:46 2022] POST /identity/v3/projects => generated 375 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:46.469305 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37acc86-765f-4bdd-bee0-42e23c09a61c service placement] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:46.480709 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a37acc86-765f-4bdd-bee0-42e23c09a61c service placement] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:46.540483 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599799f-9764-4703-a3e6-6fc77d1dcbb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:46.541104 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599799f-9764-4703-a3e6-6fc77d1dcbb6 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:46.541393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599799f-9764-4703-a3e6-6fc77d1dcbb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:46.541677 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7599799f-9764-4703-a3e6-6fc77d1dcbb6 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:46.542404 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599799f-9764-4703-a3e6-6fc77d1dcbb6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:46.544335 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7599799f-9764-4703-a3e6-6fc77d1dcbb6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:46.565468 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1357/2702] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:17:46 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:46.569556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8099b-354a-466e-8d97-dda181b6c755 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:46.583016 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb8099b-354a-466e-8d97-dda181b6c755 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:46.588051 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec6153-509a-4125-8c39-84d206947203 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:46.588744 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec6153-509a-4125-8c39-84d206947203 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:46.589048 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec6153-509a-4125-8c39-84d206947203 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:46.589321 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ec6153-509a-4125-8c39-84d206947203 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:46.590079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ec6153-509a-4125-8c39-84d206947203 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:46.591756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ec6153-509a-4125-8c39-84d206947203 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:46.606340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1346/2703] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:46.609703 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599799f-9764-4703-a3e6-6fc77d1dcbb6 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:46.621306 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7599799f-9764-4703-a3e6-6fc77d1dcbb6 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:46.629299 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a5ae558-7e1b-46ae-92fe-92e68ce899cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:46.630137 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ae558-7e1b-46ae-92fe-92e68ce899cb admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:46.630641 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ae558-7e1b-46ae-92fe-92e68ce899cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:46.630949 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a5ae558-7e1b-46ae-92fe-92e68ce899cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:46.631851 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ae558-7e1b-46ae-92fe-92e68ce899cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:46.634272 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a5ae558-7e1b-46ae-92fe-92e68ce899cb admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:46.645238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1358/2704] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:46 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:46.649858 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec6153-509a-4125-8c39-84d206947203 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:46.661199 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ec6153-509a-4125-8c39-84d206947203 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:46.666202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b257da-b830-4fa2-b195-867504fa5e42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:46.667179 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b257da-b830-4fa2-b195-867504fa5e42 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:46.667479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b257da-b830-4fa2-b195-867504fa5e42 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:46.667747 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8b257da-b830-4fa2-b195-867504fa5e42 admin admin] PATH_INFO: `/v3/projects/3ce75ff0ff394c1e957011b3f5b1b11e/users/b206a7b759164c75b1ced4bc81032cf6/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:46.685610 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b257da-b830-4fa2-b195-867504fa5e42 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ce75ff0ff394c1e957011b3f5b1b11e, user_id=b206a7b759164c75b1ced4bc81032cf6, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:46.688331 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8b257da-b830-4fa2-b195-867504fa5e42 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:46.703557 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1347/2705] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:46 2022] PUT /identity/v3/projects/3ce75ff0ff394c1e957011b3f5b1b11e/users/b206a7b759164c75b1ced4bc81032cf6/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:46.717174 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ff281-5c26-4437-acb0-dfb6b93a946e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:46.717517 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ff281-5c26-4437-acb0-dfb6b93a946e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:46.717800 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413ff281-5c26-4437-acb0-dfb6b93a946e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:46.749097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-413ff281-5c26-4437-acb0-dfb6b93a946e None None] MFA Rules not processed for user `b206a7b759164c75b1ced4bc81032cf6`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:46.750394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-413ff281-5c26-4437-acb0-dfb6b93a946e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:46.828642 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1359/2706] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:46 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:46.840209 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b257da-b830-4fa2-b195-867504fa5e42 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:46.842459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-c8b257da-b830-4fa2-b195-867504fa5e42 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:46.928221 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8b257da-b830-4fa2-b195-867504fa5e42 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:46.932198 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2c56b-7a3d-4ef5-a46f-4b5543704731 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:46.932961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2c56b-7a3d-4ef5-a46f-4b5543704731 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:46.933330 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2c56b-7a3d-4ef5-a46f-4b5543704731 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:46.933682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef2c56b-7a3d-4ef5-a46f-4b5543704731 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:46.943552 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2c56b-7a3d-4ef5-a46f-4b5543704731 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:46.945796 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef2c56b-7a3d-4ef5-a46f-4b5543704731 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:46.957239 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1348/2707] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:46 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:49.345913 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ff281-5c26-4437-acb0-dfb6b93a946e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:49.347378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-413ff281-5c26-4437-acb0-dfb6b93a946e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:49.404164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:49.419308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413ff281-5c26-4437-acb0-dfb6b93a946e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:49.424119 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a5b6d-2874-4801-91c1-7aab33b8689a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:49.425236 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a5b6d-2874-4801-91c1-7aab33b8689a service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:49.425515 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a5b6d-2874-4801-91c1-7aab33b8689a service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:49.425791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-938a5b6d-2874-4801-91c1-7aab33b8689a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:49.434659 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-938a5b6d-2874-4801-91c1-7aab33b8689a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:49.438788 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a5b6d-2874-4801-91c1-7aab33b8689a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:49.440396 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-938a5b6d-2874-4801-91c1-7aab33b8689a service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:49.450895 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1360/2708] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:49 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:52.806123 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2c56b-7a3d-4ef5-a46f-4b5543704731 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:52.807850 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fef2c56b-7a3d-4ef5-a46f-4b5543704731 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:52.852889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:52.893052 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef2c56b-7a3d-4ef5-a46f-4b5543704731 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:52.898816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf302ac-67eb-4dec-a9b9-aca06cc7c755 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:52.899470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf302ac-67eb-4dec-a9b9-aca06cc7c755 service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:52.899753 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf302ac-67eb-4dec-a9b9-aca06cc7c755 service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:52.900030 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf302ac-67eb-4dec-a9b9-aca06cc7c755 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:52.901815 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8bf302ac-67eb-4dec-a9b9-aca06cc7c755 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:52.990887 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf302ac-67eb-4dec-a9b9-aca06cc7c755 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:52.993028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf302ac-67eb-4dec-a9b9-aca06cc7c755 service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:53.015621 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1349/2709] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:52 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 210 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:56.697280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a5b6d-2874-4801-91c1-7aab33b8689a service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:56.714632 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-938a5b6d-2874-4801-91c1-7aab33b8689a service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:56.721921 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14468f28-f523-432c-be2a-a49700770cd2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:56.722605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14468f28-f523-432c-be2a-a49700770cd2 service nova] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:56.722934 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14468f28-f523-432c-be2a-a49700770cd2 service nova] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:56.723332 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14468f28-f523-432c-be2a-a49700770cd2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:56.725271 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-14468f28-f523-432c-be2a-a49700770cd2 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:56.739926 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14468f28-f523-432c-be2a-a49700770cd2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:56.741951 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14468f28-f523-432c-be2a-a49700770cd2 service nova] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:56.755912 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1361/2710] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:56 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 59 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:56.934930 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf302ac-67eb-4dec-a9b9-aca06cc7c755 service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:57.002249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf302ac-67eb-4dec-a9b9-aca06cc7c755 service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:57.009027 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d8f86-eeb1-440b-b9d5-d36558daaee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:57.009916 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d8f86-eeb1-440b-b9d5-d36558daaee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:57.010196 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d8f86-eeb1-440b-b9d5-d36558daaee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:57.010510 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7d8f86-eeb1-440b-b9d5-d36558daaee2 admin admin] PATH_INFO: `/v3/users/b206a7b759164c75b1ced4bc81032cf6` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:57.011712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d8f86-eeb1-440b-b9d5-d36558daaee2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b206a7b759164c75b1ced4bc81032cf6)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:57.013701 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7d8f86-eeb1-440b-b9d5-d36558daaee2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:57.067230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5b7d8f86-eeb1-440b-b9d5-d36558daaee2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b206a7b759164c75b1ced4bc81032cf6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:57.067907 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5b7d8f86-eeb1-440b-b9d5-d36558daaee2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b206a7b759164c75b1ced4bc81032cf6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:57.069743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=93,keystone:DELETE=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:57.079294 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5b7d8f86-eeb1-440b-b9d5-d36558daaee2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b206a7b759164c75b1ced4bc81032cf6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:57.080476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5b7d8f86-eeb1-440b-b9d5-d36558daaee2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b206a7b759164c75b1ced4bc81032cf6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:57.093097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-5b7d8f86-eeb1-440b-b9d5-d36558daaee2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b206a7b759164c75b1ced4bc81032cf6'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:57.106111 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1350/2711] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:17:56 2022] DELETE /identity/v3/users/b206a7b759164c75b1ced4bc81032cf6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:57.110583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249f663-e1e9-419b-a801-8c39dceab6a5 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:57.110882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249f663-e1e9-419b-a801-8c39dceab6a5 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:57.111168 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2249f663-e1e9-419b-a801-8c39dceab6a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:57.129622 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-2249f663-e1e9-419b-a801-8c39dceab6a5 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:57.130975 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2249f663-e1e9-419b-a801-8c39dceab6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:57.193686 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1362/2712] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:57.203331 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d8f86-eeb1-440b-b9d5-d36558daaee2 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:57.205034 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-5b7d8f86-eeb1-440b-b9d5-d36558daaee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:57.265925 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7d8f86-eeb1-440b-b9d5-d36558daaee2 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:57.269655 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8fcc7-c5c7-4407-bf5f-c5898d4abf8d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:57.270327 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8fcc7-c5c7-4407-bf5f-c5898d4abf8d service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:57.270619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8fcc7-c5c7-4407-bf5f-c5898d4abf8d service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:57.270899 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cea8fcc7-c5c7-4407-bf5f-c5898d4abf8d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:57.280712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8fcc7-c5c7-4407-bf5f-c5898d4abf8d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:57.282129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cea8fcc7-c5c7-4407-bf5f-c5898d4abf8d service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:57.291635 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1351/2713] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:17:57.536927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249f663-e1e9-419b-a801-8c39dceab6a5 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:57.538455 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2249f663-e1e9-419b-a801-8c39dceab6a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:57.553171 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2249f663-e1e9-419b-a801-8c39dceab6a5 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:57.557717 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a78e76-cc87-4bba-936d-644fc2c94054 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:57.558438 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a78e76-cc87-4bba-936d-644fc2c94054 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:57.558824 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a78e76-cc87-4bba-936d-644fc2c94054 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:57.559234 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a78e76-cc87-4bba-936d-644fc2c94054 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:57.561266 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b8a78e76-cc87-4bba-936d-644fc2c94054 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:57.566826 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8fcc7-c5c7-4407-bf5f-c5898d4abf8d service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:57.568366 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cea8fcc7-c5c7-4407-bf5f-c5898d4abf8d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:57.582889 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a78e76-cc87-4bba-936d-644fc2c94054 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:57.583654 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cea8fcc7-c5c7-4407-bf5f-c5898d4abf8d service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:57.584633 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a78e76-cc87-4bba-936d-644fc2c94054 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:57.589573 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a37941d-f2ce-4238-9784-f0e28936faa3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:57.590508 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37941d-f2ce-4238-9784-f0e28936faa3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:57.590794 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37941d-f2ce-4238-9784-f0e28936faa3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:57.591075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a37941d-f2ce-4238-9784-f0e28936faa3 admin admin] PATH_INFO: `/v3/projects/3ce75ff0ff394c1e957011b3f5b1b11e` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:57.592881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a37941d-f2ce-4238-9784-f0e28936faa3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ce75ff0ff394c1e957011b3f5b1b11e)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:57.594847 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a37941d-f2ce-4238-9784-f0e28936faa3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:57.597731 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1363/2714] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 61 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:17:57.649573 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0a37941d-f2ce-4238-9784-f0e28936faa3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ce75ff0ff394c1e957011b3f5b1b11e', 'request_id': 'req-0a37941d-f2ce-4238-9784-f0e28936faa3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:57.655532 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0a37941d-f2ce-4238-9784-f0e28936faa3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ce75ff0ff394c1e957011b3f5b1b11e', 'request_id': 'req-0a37941d-f2ce-4238-9784-f0e28936faa3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:57.664237 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0a37941d-f2ce-4238-9784-f0e28936faa3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ce75ff0ff394c1e957011b3f5b1b11e', 'request_id': 'req-0a37941d-f2ce-4238-9784-f0e28936faa3'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:57.676488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0a37941d-f2ce-4238-9784-f0e28936faa3 admin admin] The token cache is being invalidate because project 3ce75ff0ff394c1e957011b3f5b1b11e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:17:57.676908 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0a37941d-f2ce-4238-9784-f0e28936faa3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:57.677568 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0a37941d-f2ce-4238-9784-f0e28936faa3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:17:57.679467 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1352/2715] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:17:57 2022] DELETE /identity/v3/projects/3ce75ff0ff394c1e957011b3f5b1b11e => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:17:57.684056 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63380933-4398-4854-aa4e-21405e1d5eb7 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:57.684423 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63380933-4398-4854-aa4e-21405e1d5eb7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:57.684825 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63380933-4398-4854-aa4e-21405e1d5eb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:57.703145 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-63380933-4398-4854-aa4e-21405e1d5eb7 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:57.704435 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-63380933-4398-4854-aa4e-21405e1d5eb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:57.767258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1364/2716] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:57.775684 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532c7bf-51fb-4497-833e-5c972dea16cb None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:57.775972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532c7bf-51fb-4497-833e-5c972dea16cb None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:57.776255 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532c7bf-51fb-4497-833e-5c972dea16cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:57.796255 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3532c7bf-51fb-4497-833e-5c972dea16cb None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:57.797484 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3532c7bf-51fb-4497-833e-5c972dea16cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:57.805895 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1353/2717] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:57.810358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63380933-4398-4854-aa4e-21405e1d5eb7 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:57.820274 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63380933-4398-4854-aa4e-21405e1d5eb7 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:57.824952 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54dc1fb-0aa0-43ec-84f3-c01cf749d845 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:57.825623 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54dc1fb-0aa0-43ec-84f3-c01cf749d845 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:57.825907 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54dc1fb-0aa0-43ec-84f3-c01cf749d845 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:57.826186 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54dc1fb-0aa0-43ec-84f3-c01cf749d845 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:57.826968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54dc1fb-0aa0-43ec-84f3-c01cf749d845 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:17:57.827305 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54dc1fb-0aa0-43ec-84f3-c01cf749d845 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:57.828903 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54dc1fb-0aa0-43ec-84f3-c01cf749d845 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:57.841583 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1365/2718] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:17:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:17:57.846084 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532c7bf-51fb-4497-833e-5c972dea16cb None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:57.856074 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532c7bf-51fb-4497-833e-5c972dea16cb None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:57.862828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decc2b2-bc31-4661-a7dd-8dca4ecd5f79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:57.863499 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decc2b2-bc31-4661-a7dd-8dca4ecd5f79 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:57.863787 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decc2b2-bc31-4661-a7dd-8dca4ecd5f79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:57.864061 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4decc2b2-bc31-4661-a7dd-8dca4ecd5f79 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:57.864771 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4decc2b2-bc31-4661-a7dd-8dca4ecd5f79 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:57.866751 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4decc2b2-bc31-4661-a7dd-8dca4ecd5f79 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:57.895791 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1354/2719] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:17:57 2022] POST /identity/v3/projects => generated 391 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:57.897364 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54dc1fb-0aa0-43ec-84f3-c01cf749d845 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:57.908239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54dc1fb-0aa0-43ec-84f3-c01cf749d845 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:57.955269 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:57.977505 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdaaa8-35f2-47c3-8dbc-deb89523e795 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:57.978194 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdaaa8-35f2-47c3-8dbc-deb89523e795 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:57.978512 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdaaa8-35f2-47c3-8dbc-deb89523e795 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:57.978796 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebdaaa8-35f2-47c3-8dbc-deb89523e795 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:57.979511 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdaaa8-35f2-47c3-8dbc-deb89523e795 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:57.981441 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebdaaa8-35f2-47c3-8dbc-deb89523e795 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:58.002181 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1366/2720] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:17:57 2022] POST /identity/v3/users => generated 296 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:17:58.006046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decc2b2-bc31-4661-a7dd-8dca4ecd5f79 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:58.018009 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4decc2b2-bc31-4661-a7dd-8dca4ecd5f79 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:58.024457 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8609c-3891-47aa-a9ec-1a17c7a76d08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:58.025172 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8609c-3891-47aa-a9ec-1a17c7a76d08 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:58.025522 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8609c-3891-47aa-a9ec-1a17c7a76d08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:58.025894 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0a8609c-3891-47aa-a9ec-1a17c7a76d08 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:58.026748 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8609c-3891-47aa-a9ec-1a17c7a76d08 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:58.028419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0a8609c-3891-47aa-a9ec-1a17c7a76d08 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:58.039729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1355/2721] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:17:58.044154 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdaaa8-35f2-47c3-8dbc-deb89523e795 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:58.057525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebdaaa8-35f2-47c3-8dbc-deb89523e795 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:58.062342 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81142632-36de-434b-bd9f-f0aa6d585fd5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:58.062965 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81142632-36de-434b-bd9f-f0aa6d585fd5 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:58.063252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81142632-36de-434b-bd9f-f0aa6d585fd5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:58.063532 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81142632-36de-434b-bd9f-f0aa6d585fd5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:58.064269 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81142632-36de-434b-bd9f-f0aa6d585fd5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:58.065875 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81142632-36de-434b-bd9f-f0aa6d585fd5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:58.084331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1367/2722] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:17:58 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:17:58.086135 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8609c-3891-47aa-a9ec-1a17c7a76d08 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:58.096066 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0a8609c-3891-47aa-a9ec-1a17c7a76d08 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:58.100874 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d6f4a-0db4-4568-9e96-8709699c446c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:58.101789 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d6f4a-0db4-4568-9e96-8709699c446c admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:58.102072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d6f4a-0db4-4568-9e96-8709699c446c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:58.102409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d6f4a-0db4-4568-9e96-8709699c446c admin admin] PATH_INFO: `/v3/projects/4cdede8268f746daa1be618a34f54143/users/fb98153d946f4e41a36c94a6dd19ce39/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:58.111117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:INSERT=3,keystone:DELETE=10,keystone:SELECT=87 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:17:58.120214 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d6f4a-0db4-4568-9e96-8709699c446c admin admin] RBAC: Authorizing `identity:create_grant(project_id=4cdede8268f746daa1be618a34f54143, user_id=fb98153d946f4e41a36c94a6dd19ce39, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:58.122954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d6f4a-0db4-4568-9e96-8709699c446c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:58.138570 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1356/2723] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:17:58 2022] PUT /identity/v3/projects/4cdede8268f746daa1be618a34f54143/users/fb98153d946f4e41a36c94a6dd19ce39/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:17:58.149710 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97e518-4d65-4e84-895c-c030d787c0d7 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:58.150011 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97e518-4d65-4e84-895c-c030d787c0d7 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:58.150340 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97e518-4d65-4e84-895c-c030d787c0d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:58.177250 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-dc97e518-4d65-4e84-895c-c030d787c0d7 None None] MFA Rules not processed for user `fb98153d946f4e41a36c94a6dd19ce39`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:17:58.178674 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dc97e518-4d65-4e84-895c-c030d787c0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:58.258562 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1368/2724] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:17:58 2022] POST /identity/v3/auth/tokens => generated 3258 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:17:58.269478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d6f4a-0db4-4568-9e96-8709699c446c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:17:58.271226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7d4d6f4a-0db4-4568-9e96-8709699c446c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:17:58.331963 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d6f4a-0db4-4568-9e96-8709699c446c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:17:58.336316 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b82f5b-fb54-4072-a1c4-98529908c8d8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:17:58.336970 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b82f5b-fb54-4072-a1c4-98529908c8d8 service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:17:58.337255 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b82f5b-fb54-4072-a1c4-98529908c8d8 service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:17:58.337530 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b82f5b-fb54-4072-a1c4-98529908c8d8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:17:58.351464 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b82f5b-fb54-4072-a1c4-98529908c8d8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:17:58.352937 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b82f5b-fb54-4072-a1c4-98529908c8d8 service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:17:58.364260 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1357/2725] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:17:58 2022] GET /identity/v3/auth/tokens => generated 3258 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:18:00.697604 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97e518-4d65-4e84-895c-c030d787c0d7 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:00.699129 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dc97e518-4d65-4e84-895c-c030d787c0d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:00.776769 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97e518-4d65-4e84-895c-c030d787c0d7 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:00.781303 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef15d5d4-0c74-4ef8-88ef-5426630ab64f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:00.782178 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15d5d4-0c74-4ef8-88ef-5426630ab64f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:00.782501 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15d5d4-0c74-4ef8-88ef-5426630ab64f admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:00.782817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef15d5d4-0c74-4ef8-88ef-5426630ab64f admin admin] PATH_INFO: `/v3/users/98143abd2e3c47e3921184a1c5e05a89` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:00.784020 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15d5d4-0c74-4ef8-88ef-5426630ab64f admin admin] RBAC: Authorizing `identity:delete_user(user_id=98143abd2e3c47e3921184a1c5e05a89)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:00.785950 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef15d5d4-0c74-4ef8-88ef-5426630ab64f admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:00.834067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ef15d5d4-0c74-4ef8-88ef-5426630ab64f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98143abd2e3c47e3921184a1c5e05a89'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:00.841093 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:INSERT=3,keystone:DELETE=8 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:00.847534 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ef15d5d4-0c74-4ef8-88ef-5426630ab64f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98143abd2e3c47e3921184a1c5e05a89'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:00.856624 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ef15d5d4-0c74-4ef8-88ef-5426630ab64f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98143abd2e3c47e3921184a1c5e05a89'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:00.857347 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ef15d5d4-0c74-4ef8-88ef-5426630ab64f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98143abd2e3c47e3921184a1c5e05a89'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:00.864832 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-ef15d5d4-0c74-4ef8-88ef-5426630ab64f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98143abd2e3c47e3921184a1c5e05a89'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:00.871401 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1369/2726] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:18:00 2022] DELETE /identity/v3/users/98143abd2e3c47e3921184a1c5e05a89 => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:01.052644 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b82f5b-fb54-4072-a1c4-98529908c8d8 service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:01.071875 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b82f5b-fb54-4072-a1c4-98529908c8d8 service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:01.143356 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7061140d-8c90-4981-89f8-4c89f3acbbcd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:01.144236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061140d-8c90-4981-89f8-4c89f3acbbcd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:01.144717 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061140d-8c90-4981-89f8-4c89f3acbbcd admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:01.145028 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7061140d-8c90-4981-89f8-4c89f3acbbcd admin admin] PATH_INFO: `/v3/projects/beef80716d254a03a5e673b9cfd781ce` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:01.146225 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061140d-8c90-4981-89f8-4c89f3acbbcd admin admin] RBAC: Authorizing `identity:delete_project(project_id=beef80716d254a03a5e673b9cfd781ce)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:01.148225 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7061140d-8c90-4981-89f8-4c89f3acbbcd admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:01.197909 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7061140d-8c90-4981-89f8-4c89f3acbbcd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'beef80716d254a03a5e673b9cfd781ce', 'request_id': 'req-7061140d-8c90-4981-89f8-4c89f3acbbcd'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:01.203438 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7061140d-8c90-4981-89f8-4c89f3acbbcd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'beef80716d254a03a5e673b9cfd781ce', 'request_id': 'req-7061140d-8c90-4981-89f8-4c89f3acbbcd'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:01.210733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7061140d-8c90-4981-89f8-4c89f3acbbcd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'beef80716d254a03a5e673b9cfd781ce', 'request_id': 'req-7061140d-8c90-4981-89f8-4c89f3acbbcd'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:01.221978 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7061140d-8c90-4981-89f8-4c89f3acbbcd admin admin] The token cache is being invalidate because project beef80716d254a03a5e673b9cfd781ce was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:18:01.222271 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7061140d-8c90-4981-89f8-4c89f3acbbcd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:01.222939 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-7061140d-8c90-4981-89f8-4c89f3acbbcd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:01.224488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1358/2727] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:18:01 2022] DELETE /identity/v3/projects/beef80716d254a03a5e673b9cfd781ce => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:01.230001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc26cc0-2f6e-4403-b599-8a3563c21be3 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:01.230290 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc26cc0-2f6e-4403-b599-8a3563c21be3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:01.230589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc26cc0-2f6e-4403-b599-8a3563c21be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:01.249066 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-0bc26cc0-2f6e-4403-b599-8a3563c21be3 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:01.250262 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0bc26cc0-2f6e-4403-b599-8a3563c21be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:01.308413 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1370/2728] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:01.316288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcad70-51c3-4e67-a370-74699c12419d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:01.316605 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcad70-51c3-4e67-a370-74699c12419d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:01.316896 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84fcad70-51c3-4e67-a370-74699c12419d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:01.335673 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-84fcad70-51c3-4e67-a370-74699c12419d None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:01.336891 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-84fcad70-51c3-4e67-a370-74699c12419d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:01.345365 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1359/2729] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:01.349402 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc26cc0-2f6e-4403-b599-8a3563c21be3 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:01.359532 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc26cc0-2f6e-4403-b599-8a3563c21be3 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:01.364076 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add83da-fb7c-4ea7-a049-41493417d802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:01.364723 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add83da-fb7c-4ea7-a049-41493417d802 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:01.365004 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add83da-fb7c-4ea7-a049-41493417d802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:01.365280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2add83da-fb7c-4ea7-a049-41493417d802 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:01.366002 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add83da-fb7c-4ea7-a049-41493417d802 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:18:01.366324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add83da-fb7c-4ea7-a049-41493417d802 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:01.367948 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2add83da-fb7c-4ea7-a049-41493417d802 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:01.379526 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1371/2730] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:18:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:18:01.383709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcad70-51c3-4e67-a370-74699c12419d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:01.395331 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84fcad70-51c3-4e67-a370-74699c12419d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:01.405121 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6195aa-d8f1-40d4-953f-2f7dc860b879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:01.405796 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6195aa-d8f1-40d4-953f-2f7dc860b879 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:01.406085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6195aa-d8f1-40d4-953f-2f7dc860b879 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:01.406402 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f6195aa-d8f1-40d4-953f-2f7dc860b879 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:01.407127 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6195aa-d8f1-40d4-953f-2f7dc860b879 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:01.409045 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f6195aa-d8f1-40d4-953f-2f7dc860b879 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:01.420817 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:01.439346 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1360/2731] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:18:01 2022] POST /identity/v3/projects => generated 365 bytes in 56 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:18:01.443981 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add83da-fb7c-4ea7-a049-41493417d802 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:01.454088 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2add83da-fb7c-4ea7-a049-41493417d802 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:01.539146 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d96161-a2ab-4722-a4ed-fbbd7cbfd2a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:01.539818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d96161-a2ab-4722-a4ed-fbbd7cbfd2a2 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:01.540100 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d96161-a2ab-4722-a4ed-fbbd7cbfd2a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:01.540375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d96161-a2ab-4722-a4ed-fbbd7cbfd2a2 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:01.541068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d96161-a2ab-4722-a4ed-fbbd7cbfd2a2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:01.543000 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d96161-a2ab-4722-a4ed-fbbd7cbfd2a2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:01.562784 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1372/2732] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:18:01 2022] POST /identity/v3/users => generated 283 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:18:01.567171 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6195aa-d8f1-40d4-953f-2f7dc860b879 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:01.577621 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f6195aa-d8f1-40d4-953f-2f7dc860b879 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:01.582608 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d13716-9fe1-49ae-b48b-32d166a162ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:01.583196 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d13716-9fe1-49ae-b48b-32d166a162ee admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:01.583478 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d13716-9fe1-49ae-b48b-32d166a162ee admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:01.583769 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5d13716-9fe1-49ae-b48b-32d166a162ee admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:01.584495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d13716-9fe1-49ae-b48b-32d166a162ee admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:01.586629 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5d13716-9fe1-49ae-b48b-32d166a162ee admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:01.596553 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1361/2733] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:18:01.600408 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d96161-a2ab-4722-a4ed-fbbd7cbfd2a2 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:01.610939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d96161-a2ab-4722-a4ed-fbbd7cbfd2a2 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:01.615492 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1dcde8c-a17e-4b9c-8abe-80c74b508bc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:01.616142 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dcde8c-a17e-4b9c-8abe-80c74b508bc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:01.616484 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dcde8c-a17e-4b9c-8abe-80c74b508bc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:01.616755 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1dcde8c-a17e-4b9c-8abe-80c74b508bc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:01.617476 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dcde8c-a17e-4b9c-8abe-80c74b508bc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:01.619088 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1dcde8c-a17e-4b9c-8abe-80c74b508bc3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:01.628843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1373/2734] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:18:01.632982 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d13716-9fe1-49ae-b48b-32d166a162ee admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:01.643000 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5d13716-9fe1-49ae-b48b-32d166a162ee admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:01.648208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa6677-cf44-4b64-a08d-36e81e1e95c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:01.649098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa6677-cf44-4b64-a08d-36e81e1e95c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:01.649385 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa6677-cf44-4b64-a08d-36e81e1e95c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:01.649667 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68fa6677-cf44-4b64-a08d-36e81e1e95c7 admin admin] PATH_INFO: `/v3/projects/bb84847d58a148c18c60214344be1ed9/users/9cddd2e5d80a42ef87f7e84149ca1fd4/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:01.665176 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa6677-cf44-4b64-a08d-36e81e1e95c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bb84847d58a148c18c60214344be1ed9, user_id=9cddd2e5d80a42ef87f7e84149ca1fd4, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:01.667984 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68fa6677-cf44-4b64-a08d-36e81e1e95c7 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:01.681161 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1362/2735] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:18:01 2022] PUT /identity/v3/projects/bb84847d58a148c18c60214344be1ed9/users/9cddd2e5d80a42ef87f7e84149ca1fd4/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:01.691885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d07b5-b9c0-461e-a955-8274eea10e64 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:01.692177 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d07b5-b9c0-461e-a955-8274eea10e64 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:01.692457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-427d07b5-b9c0-461e-a955-8274eea10e64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:01.718845 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-427d07b5-b9c0-461e-a955-8274eea10e64 None None] MFA Rules not processed for user `9cddd2e5d80a42ef87f7e84149ca1fd4`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:01.720045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-427d07b5-b9c0-461e-a955-8274eea10e64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:01.749891 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:01.790264 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1374/2736] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:01 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:18:01.794863 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa6677-cf44-4b64-a08d-36e81e1e95c7 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:01.858045 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68fa6677-cf44-4b64-a08d-36e81e1e95c7 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:01.862789 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13655d-b060-4097-b741-48b727e73925 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:01.863453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13655d-b060-4097-b741-48b727e73925 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:01.863743 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13655d-b060-4097-b741-48b727e73925 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:01.864026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee13655d-b060-4097-b741-48b727e73925 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:01.864729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13655d-b060-4097-b741-48b727e73925 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:01.867319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee13655d-b060-4097-b741-48b727e73925 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:01.893877 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1363/2737] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:18:01 2022] POST /identity/v3/projects => generated 363 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:18:01.898195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d07b5-b9c0-461e-a955-8274eea10e64 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:01.909067 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-427d07b5-b9c0-461e-a955-8274eea10e64 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:01.965368 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1704cf3-d646-4c43-93fe-e21932d10102 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:01.966048 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1704cf3-d646-4c43-93fe-e21932d10102 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:01.966358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1704cf3-d646-4c43-93fe-e21932d10102 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:01.966640 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1704cf3-d646-4c43-93fe-e21932d10102 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:01.967337 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1704cf3-d646-4c43-93fe-e21932d10102 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:01.969302 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1704cf3-d646-4c43-93fe-e21932d10102 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:01.994495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1375/2738] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:18:01 2022] POST /identity/v3/users => generated 288 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:18:01.995684 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13655d-b060-4097-b741-48b727e73925 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:02.006049 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee13655d-b060-4097-b741-48b727e73925 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:02.010969 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e68518-ce14-4465-a28f-678fc783b435 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:02.011667 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e68518-ce14-4465-a28f-678fc783b435 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:02.011955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e68518-ce14-4465-a28f-678fc783b435 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:02.012230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e68518-ce14-4465-a28f-678fc783b435 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:02.013002 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e68518-ce14-4465-a28f-678fc783b435 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:02.014614 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e68518-ce14-4465-a28f-678fc783b435 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:02.025146 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1364/2739] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:18:02.029208 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1704cf3-d646-4c43-93fe-e21932d10102 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:02.039673 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1704cf3-d646-4c43-93fe-e21932d10102 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:02.044380 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f86288-3179-4d35-8756-c5533b8342a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:02.045257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f86288-3179-4d35-8756-c5533b8342a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:02.045540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f86288-3179-4d35-8756-c5533b8342a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:02.045817 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f86288-3179-4d35-8756-c5533b8342a5 admin admin] PATH_INFO: `/v3/projects/f5c9425633fa4188b307afe0223269da/users/72a98d7a4f0c45dbaca5e06e0fb3347b/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:02.062813 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f86288-3179-4d35-8756-c5533b8342a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5c9425633fa4188b307afe0223269da, user_id=72a98d7a4f0c45dbaca5e06e0fb3347b, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:02.065515 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46f86288-3179-4d35-8756-c5533b8342a5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:02.082718 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1376/2740] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:18:02 2022] PUT /identity/v3/projects/f5c9425633fa4188b307afe0223269da/users/72a98d7a4f0c45dbaca5e06e0fb3347b/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:02.087221 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e68518-ce14-4465-a28f-678fc783b435 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:02.097411 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e68518-ce14-4465-a28f-678fc783b435 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:02.155275 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5d8c2a-accd-4e86-a13f-ed89da750be5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:02.155941 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d8c2a-accd-4e86-a13f-ed89da750be5 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:02.156228 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d8c2a-accd-4e86-a13f-ed89da750be5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:02.156501 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5d8c2a-accd-4e86-a13f-ed89da750be5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:02.157220 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d8c2a-accd-4e86-a13f-ed89da750be5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:02.158849 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5d8c2a-accd-4e86-a13f-ed89da750be5 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:02.168584 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1365/2741] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:02 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:18:02.172943 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f86288-3179-4d35-8756-c5533b8342a5 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:02.183986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f86288-3179-4d35-8756-c5533b8342a5 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:02.188540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e364a1-1071-45c2-93e0-e3a255278b49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:02.189411 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e364a1-1071-45c2-93e0-e3a255278b49 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:02.189691 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e364a1-1071-45c2-93e0-e3a255278b49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:02.189965 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e364a1-1071-45c2-93e0-e3a255278b49 admin admin] PATH_INFO: `/v3/projects/f5c9425633fa4188b307afe0223269da/users/72a98d7a4f0c45dbaca5e06e0fb3347b/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:02.191956 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e364a1-1071-45c2-93e0-e3a255278b49 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f5c9425633fa4188b307afe0223269da, user_id=72a98d7a4f0c45dbaca5e06e0fb3347b, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:02.195222 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e364a1-1071-45c2-93e0-e3a255278b49 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:02.205257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1377/2742] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:18:02 2022] PUT /identity/v3/projects/f5c9425633fa4188b307afe0223269da/users/72a98d7a4f0c45dbaca5e06e0fb3347b/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:18:02.215571 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cb552-fd82-4664-b44a-a3466e34d9ca None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:02.215856 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cb552-fd82-4664-b44a-a3466e34d9ca None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:02.216128 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5cb552-fd82-4664-b44a-a3466e34d9ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:02.242145 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7a5cb552-fd82-4664-b44a-a3466e34d9ca None None] MFA Rules not processed for user `72a98d7a4f0c45dbaca5e06e0fb3347b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:02.243378 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7a5cb552-fd82-4664-b44a-a3466e34d9ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:02.255816 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:02.311828 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1366/2743] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:02 2022] POST /identity/v3/auth/tokens => generated 3297 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:02.323428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e364a1-1071-45c2-93e0-e3a255278b49 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:02.325118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d0e364a1-1071-45c2-93e0-e3a255278b49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:02.384342 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e364a1-1071-45c2-93e0-e3a255278b49 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:02.388001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae384ffb-e9fa-4a0a-bb58-ed10b979f76c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:02.389219 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae384ffb-e9fa-4a0a-bb58-ed10b979f76c service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:02.389510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae384ffb-e9fa-4a0a-bb58-ed10b979f76c service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:02.389787 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae384ffb-e9fa-4a0a-bb58-ed10b979f76c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:02.398709 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae384ffb-e9fa-4a0a-bb58-ed10b979f76c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:02.400127 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae384ffb-e9fa-4a0a-bb58-ed10b979f76c service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:02.409575 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1378/2744] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:18:02 2022] GET /identity/v3/auth/tokens => generated 3297 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:18:02.805858 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cb552-fd82-4664-b44a-a3466e34d9ca None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:02.868003 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5cb552-fd82-4664-b44a-a3466e34d9ca None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:02.872569 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f439c5-4788-4eb4-8613-a03e0a372d7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:02.873429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f439c5-4788-4eb4-8613-a03e0a372d7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:02.873717 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f439c5-4788-4eb4-8613-a03e0a372d7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:02.873997 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f439c5-4788-4eb4-8613-a03e0a372d7e admin admin] PATH_INFO: `/v3/users/9cddd2e5d80a42ef87f7e84149ca1fd4` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:02.875213 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f439c5-4788-4eb4-8613-a03e0a372d7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=9cddd2e5d80a42ef87f7e84149ca1fd4)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:02.877127 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f439c5-4788-4eb4-8613-a03e0a372d7e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:02.926355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-21f439c5-4788-4eb4-8613-a03e0a372d7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9cddd2e5d80a42ef87f7e84149ca1fd4'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:02.926941 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-21f439c5-4788-4eb4-8613-a03e0a372d7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9cddd2e5d80a42ef87f7e84149ca1fd4'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:02.934508 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-21f439c5-4788-4eb4-8613-a03e0a372d7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9cddd2e5d80a42ef87f7e84149ca1fd4'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:02.935164 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-21f439c5-4788-4eb4-8613-a03e0a372d7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9cddd2e5d80a42ef87f7e84149ca1fd4'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:02.947495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-21f439c5-4788-4eb4-8613-a03e0a372d7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9cddd2e5d80a42ef87f7e84149ca1fd4'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:02.959653 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1367/2745] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:18:02 2022] DELETE /identity/v3/users/9cddd2e5d80a42ef87f7e84149ca1fd4 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:02.964782 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc116a9-1f26-44fd-95a5-5b54f158118e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:02.965280 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc116a9-1f26-44fd-95a5-5b54f158118e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:02.965518 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cc116a9-1f26-44fd-95a5-5b54f158118e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:02.983429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1cc116a9-1f26-44fd-95a5-5b54f158118e None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:02.984657 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1cc116a9-1f26-44fd-95a5-5b54f158118e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:03.004322 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:03.065775 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1379/2746] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:03.075479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f439c5-4788-4eb4-8613-a03e0a372d7e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:03.076952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-21f439c5-4788-4eb4-8613-a03e0a372d7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:03.112767 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:03.137373 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f439c5-4788-4eb4-8613-a03e0a372d7e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:03.141237 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32973f-05c2-41c0-a10e-856a1a31f9ba service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:03.141890 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32973f-05c2-41c0-a10e-856a1a31f9ba service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:03.142181 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32973f-05c2-41c0-a10e-856a1a31f9ba service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:03.142494 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec32973f-05c2-41c0-a10e-856a1a31f9ba service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:03.153132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32973f-05c2-41c0-a10e-856a1a31f9ba service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:03.154501 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec32973f-05c2-41c0-a10e-856a1a31f9ba service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:03.165149 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1368/2747] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:18:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:18:03.433684 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc116a9-1f26-44fd-95a5-5b54f158118e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:03.435065 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1cc116a9-1f26-44fd-95a5-5b54f158118e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:03.453982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cc116a9-1f26-44fd-95a5-5b54f158118e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:03.458968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62fec8b-9103-4cea-bec8-519d89fbf20c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:03.459919 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fec8b-9103-4cea-bec8-519d89fbf20c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:03.460824 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fec8b-9103-4cea-bec8-519d89fbf20c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:03.460982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62fec8b-9103-4cea-bec8-519d89fbf20c admin admin] PATH_INFO: `/v3/projects/bb84847d58a148c18c60214344be1ed9` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:03.462442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62fec8b-9103-4cea-bec8-519d89fbf20c admin admin] RBAC: Authorizing `identity:delete_project(project_id=bb84847d58a148c18c60214344be1ed9)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:03.464431 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62fec8b-9103-4cea-bec8-519d89fbf20c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:03.511808 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b62fec8b-9103-4cea-bec8-519d89fbf20c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bb84847d58a148c18c60214344be1ed9', 'request_id': 'req-b62fec8b-9103-4cea-bec8-519d89fbf20c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:03.518986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b62fec8b-9103-4cea-bec8-519d89fbf20c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bb84847d58a148c18c60214344be1ed9', 'request_id': 'req-b62fec8b-9103-4cea-bec8-519d89fbf20c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:03.525266 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b62fec8b-9103-4cea-bec8-519d89fbf20c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bb84847d58a148c18c60214344be1ed9', 'request_id': 'req-b62fec8b-9103-4cea-bec8-519d89fbf20c'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:03.536396 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b62fec8b-9103-4cea-bec8-519d89fbf20c admin admin] The token cache is being invalidate because project bb84847d58a148c18c60214344be1ed9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:18:03.536688 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b62fec8b-9103-4cea-bec8-519d89fbf20c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:03.537341 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-b62fec8b-9103-4cea-bec8-519d89fbf20c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:03.539103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1380/2748] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:18:03 2022] DELETE /identity/v3/projects/bb84847d58a148c18c60214344be1ed9 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:03.543111 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32973f-05c2-41c0-a10e-856a1a31f9ba service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:03.554070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec32973f-05c2-41c0-a10e-856a1a31f9ba service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:03.610404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58bf621d-c6fb-4c49-9cb4-33a602d5545e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:03.611287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf621d-c6fb-4c49-9cb4-33a602d5545e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:03.611574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf621d-c6fb-4c49-9cb4-33a602d5545e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:03.611854 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58bf621d-c6fb-4c49-9cb4-33a602d5545e admin admin] PATH_INFO: `/v3/users/72a98d7a4f0c45dbaca5e06e0fb3347b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:03.613059 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf621d-c6fb-4c49-9cb4-33a602d5545e admin admin] RBAC: Authorizing `identity:delete_user(user_id=72a98d7a4f0c45dbaca5e06e0fb3347b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:03.615062 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58bf621d-c6fb-4c49-9cb4-33a602d5545e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:03.662040 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-58bf621d-c6fb-4c49-9cb4-33a602d5545e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72a98d7a4f0c45dbaca5e06e0fb3347b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:03.662711 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-58bf621d-c6fb-4c49-9cb4-33a602d5545e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72a98d7a4f0c45dbaca5e06e0fb3347b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:03.669965 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-58bf621d-c6fb-4c49-9cb4-33a602d5545e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72a98d7a4f0c45dbaca5e06e0fb3347b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:03.670679 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-58bf621d-c6fb-4c49-9cb4-33a602d5545e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72a98d7a4f0c45dbaca5e06e0fb3347b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:03.683461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-58bf621d-c6fb-4c49-9cb4-33a602d5545e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72a98d7a4f0c45dbaca5e06e0fb3347b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:03.695963 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1369/2749] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:18:03 2022] DELETE /identity/v3/users/72a98d7a4f0c45dbaca5e06e0fb3347b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:03.971791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62fec8b-9103-4cea-bec8-519d89fbf20c admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:03.973293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-b62fec8b-9103-4cea-bec8-519d89fbf20c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:03.987312 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62fec8b-9103-4cea-bec8-519d89fbf20c admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:04.052436 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e85420ed-bc0e-4725-b5e1-007288974a4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:04.053558 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85420ed-bc0e-4725-b5e1-007288974a4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:04.053767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85420ed-bc0e-4725-b5e1-007288974a4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:04.053939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e85420ed-bc0e-4725-b5e1-007288974a4e admin admin] PATH_INFO: `/v3/projects/f5c9425633fa4188b307afe0223269da` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:04.055304 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85420ed-bc0e-4725-b5e1-007288974a4e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f5c9425633fa4188b307afe0223269da)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:04.057230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e85420ed-bc0e-4725-b5e1-007288974a4e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:04.108297 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e85420ed-bc0e-4725-b5e1-007288974a4e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f5c9425633fa4188b307afe0223269da', 'request_id': 'req-e85420ed-bc0e-4725-b5e1-007288974a4e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:04.109941 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:04.116594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e85420ed-bc0e-4725-b5e1-007288974a4e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f5c9425633fa4188b307afe0223269da', 'request_id': 'req-e85420ed-bc0e-4725-b5e1-007288974a4e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:04.123250 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e85420ed-bc0e-4725-b5e1-007288974a4e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f5c9425633fa4188b307afe0223269da', 'request_id': 'req-e85420ed-bc0e-4725-b5e1-007288974a4e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:04.138702 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e85420ed-bc0e-4725-b5e1-007288974a4e admin admin] The token cache is being invalidate because project f5c9425633fa4188b307afe0223269da was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:18:04.139133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e85420ed-bc0e-4725-b5e1-007288974a4e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:04.139984 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e85420ed-bc0e-4725-b5e1-007288974a4e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:04.141676 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1381/2750] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:18:03 2022] DELETE /identity/v3/projects/f5c9425633fa4188b307afe0223269da => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:04.146754 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e5c75-9666-4b65-be2a-000363274119 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:04.148483 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e5c75-9666-4b65-be2a-000363274119 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:04.148483 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-397e5c75-9666-4b65-be2a-000363274119 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:04.171717 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-397e5c75-9666-4b65-be2a-000363274119 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:04.173016 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-397e5c75-9666-4b65-be2a-000363274119 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:04.227479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:04.242464 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1370/2751] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:04.250183 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c2d0d-03df-4bb5-949c-931a956fdd30 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:04.250619 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c2d0d-03df-4bb5-949c-931a956fdd30 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:04.253328 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358c2d0d-03df-4bb5-949c-931a956fdd30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:04.272517 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-358c2d0d-03df-4bb5-949c-931a956fdd30 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:04.274377 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-358c2d0d-03df-4bb5-949c-931a956fdd30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:04.282416 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1382/2752] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:04.286814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e5c75-9666-4b65-be2a-000363274119 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:04.296855 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-397e5c75-9666-4b65-be2a-000363274119 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:04.302280 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66580c16-0f35-4fb5-9e64-6c431d56bb6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:04.302974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66580c16-0f35-4fb5-9e64-6c431d56bb6e admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:04.303267 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66580c16-0f35-4fb5-9e64-6c431d56bb6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:04.303549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66580c16-0f35-4fb5-9e64-6c431d56bb6e admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:04.304320 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66580c16-0f35-4fb5-9e64-6c431d56bb6e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:18:04.304634 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66580c16-0f35-4fb5-9e64-6c431d56bb6e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:04.306232 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66580c16-0f35-4fb5-9e64-6c431d56bb6e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:04.320709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1371/2753] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:18:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:18:04.323655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c2d0d-03df-4bb5-949c-931a956fdd30 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:04.333097 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358c2d0d-03df-4bb5-949c-931a956fdd30 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:04.338443 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ddfa23-8d30-4d32-a4e5-b23412179745 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:04.339102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddfa23-8d30-4d32-a4e5-b23412179745 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:04.339459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddfa23-8d30-4d32-a4e5-b23412179745 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:04.339744 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ddfa23-8d30-4d32-a4e5-b23412179745 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:04.340454 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ddfa23-8d30-4d32-a4e5-b23412179745 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:04.342457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ddfa23-8d30-4d32-a4e5-b23412179745 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:04.369524 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1383/2754] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:18:04 2022] POST /identity/v3/projects => generated 387 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:18:04.374114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66580c16-0f35-4fb5-9e64-6c431d56bb6e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:04.386245 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66580c16-0f35-4fb5-9e64-6c431d56bb6e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:04.445332 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914f2ea-71b8-48fa-95f5-d5e88c79383a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:04.446159 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914f2ea-71b8-48fa-95f5-d5e88c79383a admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:04.446483 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914f2ea-71b8-48fa-95f5-d5e88c79383a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:04.446770 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914f2ea-71b8-48fa-95f5-d5e88c79383a admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:04.447493 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914f2ea-71b8-48fa-95f5-d5e88c79383a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:04.449439 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914f2ea-71b8-48fa-95f5-d5e88c79383a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:04.475414 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1372/2755] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:18:04 2022] POST /identity/v3/users => generated 294 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:18:04.476929 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ddfa23-8d30-4d32-a4e5-b23412179745 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:04.491753 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ddfa23-8d30-4d32-a4e5-b23412179745 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:04.496969 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688efbf-9f36-4b37-a171-a6deccd1f9b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:04.497760 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688efbf-9f36-4b37-a171-a6deccd1f9b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:04.498043 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688efbf-9f36-4b37-a171-a6deccd1f9b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:04.498556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c688efbf-9f36-4b37-a171-a6deccd1f9b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:04.499295 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c688efbf-9f36-4b37-a171-a6deccd1f9b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:04.500939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c688efbf-9f36-4b37-a171-a6deccd1f9b3 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:04.515962 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1384/2756] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:18:04.520829 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914f2ea-71b8-48fa-95f5-d5e88c79383a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:04.531974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914f2ea-71b8-48fa-95f5-d5e88c79383a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:04.537355 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87bca13-6d9b-4013-b7e0-6402140990a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:04.538017 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bca13-6d9b-4013-b7e0-6402140990a0 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:04.538584 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bca13-6d9b-4013-b7e0-6402140990a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:04.538584 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87bca13-6d9b-4013-b7e0-6402140990a0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:04.539339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bca13-6d9b-4013-b7e0-6402140990a0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:04.540963 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87bca13-6d9b-4013-b7e0-6402140990a0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:04.551771 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1373/2757] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:18:04.556656 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688efbf-9f36-4b37-a171-a6deccd1f9b3 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:04.569479 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c688efbf-9f36-4b37-a171-a6deccd1f9b3 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:04.575390 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf374d1d-28ce-4eaf-9200-cf448935df20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:04.576281 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf374d1d-28ce-4eaf-9200-cf448935df20 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:04.576563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf374d1d-28ce-4eaf-9200-cf448935df20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:04.576837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf374d1d-28ce-4eaf-9200-cf448935df20 admin admin] PATH_INFO: `/v3/projects/0514eb6809254eba977e11e7145775bb/users/0ae0bfe381a64e7fbb9af5626277d411/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:04.592961 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf374d1d-28ce-4eaf-9200-cf448935df20 admin admin] RBAC: Authorizing `identity:create_grant(project_id=0514eb6809254eba977e11e7145775bb, user_id=0ae0bfe381a64e7fbb9af5626277d411, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:04.595683 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf374d1d-28ce-4eaf-9200-cf448935df20 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:04.611235 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1385/2758] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:18:04 2022] PUT /identity/v3/projects/0514eb6809254eba977e11e7145775bb/users/0ae0bfe381a64e7fbb9af5626277d411/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:04.624138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00122e-04e5-485f-8914-e165fda14520 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:04.624138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00122e-04e5-485f-8914-e165fda14520 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:04.624487 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a00122e-04e5-485f-8914-e165fda14520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:04.688169 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-7a00122e-04e5-485f-8914-e165fda14520 None None] MFA Rules not processed for user `0ae0bfe381a64e7fbb9af5626277d411`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:04.692692 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-7a00122e-04e5-485f-8914-e165fda14520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:04.785075 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1374/2759] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:04 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:04.787670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf374d1d-28ce-4eaf-9200-cf448935df20 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:04.865513 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf374d1d-28ce-4eaf-9200-cf448935df20 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:04.872172 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101da6f4-c705-4df9-acbe-46c9762b2ce1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:04.872974 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101da6f4-c705-4df9-acbe-46c9762b2ce1 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:04.873329 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101da6f4-c705-4df9-acbe-46c9762b2ce1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:04.873686 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101da6f4-c705-4df9-acbe-46c9762b2ce1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:04.874457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101da6f4-c705-4df9-acbe-46c9762b2ce1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:04.876373 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-101da6f4-c705-4df9-acbe-46c9762b2ce1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:04.914068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1386/2760] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:18:04 2022] POST /identity/v3/projects => generated 387 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:18:04.915632 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00122e-04e5-485f-8914-e165fda14520 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:04.928861 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a00122e-04e5-485f-8914-e165fda14520 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:04.987778 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:05.002424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f2bef-1a2d-4c3b-9cf8-ababa2d4d246 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:05.003100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f2bef-1a2d-4c3b-9cf8-ababa2d4d246 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:05.003384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f2bef-1a2d-4c3b-9cf8-ababa2d4d246 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:05.003657 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e6f2bef-1a2d-4c3b-9cf8-ababa2d4d246 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:05.004377 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f2bef-1a2d-4c3b-9cf8-ababa2d4d246 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:05.006264 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e6f2bef-1a2d-4c3b-9cf8-ababa2d4d246 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:05.025279 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1375/2761] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:18:04 2022] POST /identity/v3/users => generated 300 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:18:05.029299 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101da6f4-c705-4df9-acbe-46c9762b2ce1 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:05.040173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-101da6f4-c705-4df9-acbe-46c9762b2ce1 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:05.044788 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4382f3-5c35-4d82-8120-af7d205b57b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:05.045917 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4382f3-5c35-4d82-8120-af7d205b57b7 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:05.046204 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4382f3-5c35-4d82-8120-af7d205b57b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:05.046565 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4382f3-5c35-4d82-8120-af7d205b57b7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:05.047324 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4382f3-5c35-4d82-8120-af7d205b57b7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:05.049144 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4382f3-5c35-4d82-8120-af7d205b57b7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:05.061819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1387/2762] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:18:05.066287 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f2bef-1a2d-4c3b-9cf8-ababa2d4d246 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:05.082470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e6f2bef-1a2d-4c3b-9cf8-ababa2d4d246 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:05.087674 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59110741-e287-4715-a704-262373343810 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:05.088521 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59110741-e287-4715-a704-262373343810 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:05.088806 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59110741-e287-4715-a704-262373343810 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:05.089082 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59110741-e287-4715-a704-262373343810 admin admin] PATH_INFO: `/v3/projects/efc92eafa7124e18a88b51af202e5cd1/users/802931f0a2884d80a83565017907efae/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:05.104956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59110741-e287-4715-a704-262373343810 admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc92eafa7124e18a88b51af202e5cd1, user_id=802931f0a2884d80a83565017907efae, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:05.107664 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59110741-e287-4715-a704-262373343810 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:05.122131 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1376/2763] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:18:05 2022] PUT /identity/v3/projects/efc92eafa7124e18a88b51af202e5cd1/users/802931f0a2884d80a83565017907efae/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:05.126042 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4382f3-5c35-4d82-8120-af7d205b57b7 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:05.135847 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4382f3-5c35-4d82-8120-af7d205b57b7 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:05.188482 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:05.204346 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c165c856-24d0-457e-a370-22662dd1bb7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:05.205373 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165c856-24d0-457e-a370-22662dd1bb7e admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:05.205665 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165c856-24d0-457e-a370-22662dd1bb7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:05.205989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c165c856-24d0-457e-a370-22662dd1bb7e admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:05.206797 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165c856-24d0-457e-a370-22662dd1bb7e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:05.208461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c165c856-24d0-457e-a370-22662dd1bb7e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:05.222358 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1388/2764] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:05 2022] GET /identity/v3/roles => generated 1420 bytes in 96 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:18:05.226518 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59110741-e287-4715-a704-262373343810 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:05.237642 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59110741-e287-4715-a704-262373343810 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:05.245541 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945fbce-b4cb-4d2e-b19a-b5f1ec04e6bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:05.246546 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945fbce-b4cb-4d2e-b19a-b5f1ec04e6bb admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:05.247175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945fbce-b4cb-4d2e-b19a-b5f1ec04e6bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:05.247175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4945fbce-b4cb-4d2e-b19a-b5f1ec04e6bb admin admin] PATH_INFO: `/v3/projects/efc92eafa7124e18a88b51af202e5cd1/users/802931f0a2884d80a83565017907efae/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:05.249469 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945fbce-b4cb-4d2e-b19a-b5f1ec04e6bb admin admin] RBAC: Authorizing `identity:create_grant(project_id=efc92eafa7124e18a88b51af202e5cd1, user_id=802931f0a2884d80a83565017907efae, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:05.252233 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4945fbce-b4cb-4d2e-b19a-b5f1ec04e6bb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:05.264713 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1377/2765] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:18:05 2022] PUT /identity/v3/projects/efc92eafa7124e18a88b51af202e5cd1/users/802931f0a2884d80a83565017907efae/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:18:05.275399 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9294c0-a3fb-4725-9cf4-a18673970e52 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:05.275621 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9294c0-a3fb-4725-9cf4-a18673970e52 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:05.275929 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9294c0-a3fb-4725-9cf4-a18673970e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:05.303459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-0a9294c0-a3fb-4725-9cf4-a18673970e52 None None] MFA Rules not processed for user `802931f0a2884d80a83565017907efae`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:05.304692 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0a9294c0-a3fb-4725-9cf4-a18673970e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:05.375315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1389/2766] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:05 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:05.387021 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945fbce-b4cb-4d2e-b19a-b5f1ec04e6bb admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:05.388786 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-4945fbce-b4cb-4d2e-b19a-b5f1ec04e6bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:05.449795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4945fbce-b4cb-4d2e-b19a-b5f1ec04e6bb admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:05.453522 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9813b8e-fff2-461b-ad14-22202f4293de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:05.454292 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9813b8e-fff2-461b-ad14-22202f4293de service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:05.454557 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9813b8e-fff2-461b-ad14-22202f4293de service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:05.454836 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9813b8e-fff2-461b-ad14-22202f4293de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:05.465005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9813b8e-fff2-461b-ad14-22202f4293de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:05.466490 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9813b8e-fff2-461b-ad14-22202f4293de service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:05.477489 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1378/2767] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:18:05 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:18:05.505071 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9294c0-a3fb-4725-9cf4-a18673970e52 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:05.570236 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9294c0-a3fb-4725-9cf4-a18673970e52 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:05.575285 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe3487-5cf6-4592-a616-537bb8df66fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:05.576169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe3487-5cf6-4592-a616-537bb8df66fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:05.576457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe3487-5cf6-4592-a616-537bb8df66fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:05.576737 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7fe3487-5cf6-4592-a616-537bb8df66fb admin admin] PATH_INFO: `/v3/users/0ae0bfe381a64e7fbb9af5626277d411` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:05.577948 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe3487-5cf6-4592-a616-537bb8df66fb admin admin] RBAC: Authorizing `identity:delete_user(user_id=0ae0bfe381a64e7fbb9af5626277d411)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:05.579943 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7fe3487-5cf6-4592-a616-537bb8df66fb admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:05.631662 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e7fe3487-5cf6-4592-a616-537bb8df66fb admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0ae0bfe381a64e7fbb9af5626277d411'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:05.644396 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e7fe3487-5cf6-4592-a616-537bb8df66fb admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0ae0bfe381a64e7fbb9af5626277d411'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:05.653457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e7fe3487-5cf6-4592-a616-537bb8df66fb admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0ae0bfe381a64e7fbb9af5626277d411'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:05.654490 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e7fe3487-5cf6-4592-a616-537bb8df66fb admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0ae0bfe381a64e7fbb9af5626277d411'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:05.656082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:05.665356 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-e7fe3487-5cf6-4592-a616-537bb8df66fb admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0ae0bfe381a64e7fbb9af5626277d411'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:05.671581 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1390/2768] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:18:05 2022] DELETE /identity/v3/users/0ae0bfe381a64e7fbb9af5626277d411 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:18:05.676263 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fef2bd-0fdb-45c8-a5a3-44f850e7b12b None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:05.676591 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fef2bd-0fdb-45c8-a5a3-44f850e7b12b None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:05.676854 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fef2bd-0fdb-45c8-a5a3-44f850e7b12b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:05.695183 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-38fef2bd-0fdb-45c8-a5a3-44f850e7b12b None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:05.696412 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-38fef2bd-0fdb-45c8-a5a3-44f850e7b12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:05.758810 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1379/2769] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:05.768393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe3487-5cf6-4592-a616-537bb8df66fb admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:05.769988 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e7fe3487-5cf6-4592-a616-537bb8df66fb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:05.829191 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7fe3487-5cf6-4592-a616-537bb8df66fb admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:05.832745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfbf498-c644-4907-985e-1f435dfdc788 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:05.833393 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbf498-c644-4907-985e-1f435dfdc788 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:05.833676 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbf498-c644-4907-985e-1f435dfdc788 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:05.833953 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcfbf498-c644-4907-985e-1f435dfdc788 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:05.844470 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfbf498-c644-4907-985e-1f435dfdc788 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:05.845923 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcfbf498-c644-4907-985e-1f435dfdc788 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:05.856154 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1391/2770] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:18:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:18:06.144609 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fef2bd-0fdb-45c8-a5a3-44f850e7b12b None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:06.146170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-38fef2bd-0fdb-45c8-a5a3-44f850e7b12b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:06.159272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fef2bd-0fdb-45c8-a5a3-44f850e7b12b None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:06.164235 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90d010-d4ce-4e7c-8260-3f3a286e25f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:06.165218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90d010-d4ce-4e7c-8260-3f3a286e25f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:06.165606 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90d010-d4ce-4e7c-8260-3f3a286e25f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:06.165889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad90d010-d4ce-4e7c-8260-3f3a286e25f0 admin admin] PATH_INFO: `/v3/projects/0514eb6809254eba977e11e7145775bb` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:06.167258 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad90d010-d4ce-4e7c-8260-3f3a286e25f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=0514eb6809254eba977e11e7145775bb)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:06.169661 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad90d010-d4ce-4e7c-8260-3f3a286e25f0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:06.212198 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:INSERT=5,keystone:DELETE=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:06.221841 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ad90d010-d4ce-4e7c-8260-3f3a286e25f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '0514eb6809254eba977e11e7145775bb', 'request_id': 'req-ad90d010-d4ce-4e7c-8260-3f3a286e25f0'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:06.228026 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ad90d010-d4ce-4e7c-8260-3f3a286e25f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '0514eb6809254eba977e11e7145775bb', 'request_id': 'req-ad90d010-d4ce-4e7c-8260-3f3a286e25f0'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:06.235760 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ad90d010-d4ce-4e7c-8260-3f3a286e25f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '0514eb6809254eba977e11e7145775bb', 'request_id': 'req-ad90d010-d4ce-4e7c-8260-3f3a286e25f0'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:06.247622 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ad90d010-d4ce-4e7c-8260-3f3a286e25f0 admin admin] The token cache is being invalidate because project 0514eb6809254eba977e11e7145775bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:18:06.248388 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ad90d010-d4ce-4e7c-8260-3f3a286e25f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:06.248579 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-ad90d010-d4ce-4e7c-8260-3f3a286e25f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:06.250204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1380/2771] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:18:06 2022] DELETE /identity/v3/projects/0514eb6809254eba977e11e7145775bb => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:06.254514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfbf498-c644-4907-985e-1f435dfdc788 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:06.264469 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcfbf498-c644-4907-985e-1f435dfdc788 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:06.320162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c42794d6-5f03-492d-ba7c-b6935320dbe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:06.321025 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42794d6-5f03-492d-ba7c-b6935320dbe2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:06.321311 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42794d6-5f03-492d-ba7c-b6935320dbe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:06.321589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c42794d6-5f03-492d-ba7c-b6935320dbe2 admin admin] PATH_INFO: `/v3/users/802931f0a2884d80a83565017907efae` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:06.322805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42794d6-5f03-492d-ba7c-b6935320dbe2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=802931f0a2884d80a83565017907efae)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:06.324761 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c42794d6-5f03-492d-ba7c-b6935320dbe2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:06.371098 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c42794d6-5f03-492d-ba7c-b6935320dbe2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '802931f0a2884d80a83565017907efae'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:06.383164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c42794d6-5f03-492d-ba7c-b6935320dbe2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '802931f0a2884d80a83565017907efae'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:06.390637 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c42794d6-5f03-492d-ba7c-b6935320dbe2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '802931f0a2884d80a83565017907efae'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:06.391251 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c42794d6-5f03-492d-ba7c-b6935320dbe2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '802931f0a2884d80a83565017907efae'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:06.398235 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-c42794d6-5f03-492d-ba7c-b6935320dbe2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '802931f0a2884d80a83565017907efae'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:06.404195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1392/2772] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:18:06 2022] DELETE /identity/v3/users/802931f0a2884d80a83565017907efae => generated 0 bytes in 150 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:06.674902 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90d010-d4ce-4e7c-8260-3f3a286e25f0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:06.676363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ad90d010-d4ce-4e7c-8260-3f3a286e25f0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:06.690842 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad90d010-d4ce-4e7c-8260-3f3a286e25f0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:06.750165 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08e02ce-20a5-4913-bd89-49ec11f7153d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:06.751063 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e02ce-20a5-4913-bd89-49ec11f7153d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:06.751348 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e02ce-20a5-4913-bd89-49ec11f7153d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:06.751627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08e02ce-20a5-4913-bd89-49ec11f7153d admin admin] PATH_INFO: `/v3/projects/efc92eafa7124e18a88b51af202e5cd1` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:06.752800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e02ce-20a5-4913-bd89-49ec11f7153d admin admin] RBAC: Authorizing `identity:delete_project(project_id=efc92eafa7124e18a88b51af202e5cd1)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:06.754985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08e02ce-20a5-4913-bd89-49ec11f7153d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:06.802573 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c08e02ce-20a5-4913-bd89-49ec11f7153d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'efc92eafa7124e18a88b51af202e5cd1', 'request_id': 'req-c08e02ce-20a5-4913-bd89-49ec11f7153d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:06.807952 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c08e02ce-20a5-4913-bd89-49ec11f7153d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'efc92eafa7124e18a88b51af202e5cd1', 'request_id': 'req-c08e02ce-20a5-4913-bd89-49ec11f7153d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:06.815110 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c08e02ce-20a5-4913-bd89-49ec11f7153d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'efc92eafa7124e18a88b51af202e5cd1', 'request_id': 'req-c08e02ce-20a5-4913-bd89-49ec11f7153d'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:06.826416 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c08e02ce-20a5-4913-bd89-49ec11f7153d admin admin] The token cache is being invalidate because project efc92eafa7124e18a88b51af202e5cd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:18:06.826706 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c08e02ce-20a5-4913-bd89-49ec11f7153d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:06.827329 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-c08e02ce-20a5-4913-bd89-49ec11f7153d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:06.828879 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1381/2773] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:18:06 2022] DELETE /identity/v3/projects/efc92eafa7124e18a88b51af202e5cd1 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:06.833736 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860439b1-6863-4530-9a50-b6477599e52e None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:06.834035 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860439b1-6863-4530-9a50-b6477599e52e None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:06.834348 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-860439b1-6863-4530-9a50-b6477599e52e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:06.853700 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-860439b1-6863-4530-9a50-b6477599e52e None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:06.854956 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-860439b1-6863-4530-9a50-b6477599e52e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:06.860413 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:06.916668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1393/2774] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:06.925023 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f55062-ff0c-4f90-9c36-97178866cafd None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:06.925243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f55062-ff0c-4f90-9c36-97178866cafd None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:06.925523 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f55062-ff0c-4f90-9c36-97178866cafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:06.944364 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-30f55062-ff0c-4f90-9c36-97178866cafd None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:06.945573 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-30f55062-ff0c-4f90-9c36-97178866cafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:06.953601 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1382/2775] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:06.957778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860439b1-6863-4530-9a50-b6477599e52e None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:06.968033 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-860439b1-6863-4530-9a50-b6477599e52e None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:06.972672 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2e2b9-b468-411c-abb7-19f875bcb23d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:06.973333 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2e2b9-b468-411c-abb7-19f875bcb23d admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:06.973621 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2e2b9-b468-411c-abb7-19f875bcb23d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:06.973901 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c2e2b9-b468-411c-abb7-19f875bcb23d admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:06.974693 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2e2b9-b468-411c-abb7-19f875bcb23d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:18:06.975007 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2e2b9-b468-411c-abb7-19f875bcb23d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:06.977258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c2e2b9-b468-411c-abb7-19f875bcb23d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:06.988762 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1394/2776] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:18:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:18:06.992992 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f55062-ff0c-4f90-9c36-97178866cafd None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:07.002261 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f55062-ff0c-4f90-9c36-97178866cafd None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:07.007418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7471a7-7812-49de-ba8c-2e5c639c9881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:07.008090 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7471a7-7812-49de-ba8c-2e5c639c9881 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:07.008395 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7471a7-7812-49de-ba8c-2e5c639c9881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:07.008649 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d7471a7-7812-49de-ba8c-2e5c639c9881 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:07.009338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7471a7-7812-49de-ba8c-2e5c639c9881 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:07.011326 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d7471a7-7812-49de-ba8c-2e5c639c9881 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:07.037156 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1383/2777] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:18:06 2022] POST /identity/v3/projects => generated 387 bytes in 45 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:18:07.041297 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2e2b9-b468-411c-abb7-19f875bcb23d admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:07.051844 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c2e2b9-b468-411c-abb7-19f875bcb23d admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:07.109877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f784da-e67d-479c-bda3-4b5f5568bed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:07.110592 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f784da-e67d-479c-bda3-4b5f5568bed6 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:07.110880 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f784da-e67d-479c-bda3-4b5f5568bed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:07.111162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f784da-e67d-479c-bda3-4b5f5568bed6 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:07.111890 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f784da-e67d-479c-bda3-4b5f5568bed6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:07.113833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f784da-e67d-479c-bda3-4b5f5568bed6 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:07.134121 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1395/2778] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:18:07 2022] POST /identity/v3/users => generated 294 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:18:07.138266 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7471a7-7812-49de-ba8c-2e5c639c9881 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:07.149731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d7471a7-7812-49de-ba8c-2e5c639c9881 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:07.154815 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5fcc2-06bd-46ac-9931-940859d21940 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:07.155474 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5fcc2-06bd-46ac-9931-940859d21940 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:07.155763 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5fcc2-06bd-46ac-9931-940859d21940 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:07.156071 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11e5fcc2-06bd-46ac-9931-940859d21940 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:07.156821 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5fcc2-06bd-46ac-9931-940859d21940 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:07.159885 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11e5fcc2-06bd-46ac-9931-940859d21940 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:07.170204 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1384/2779] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:18:07.174240 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f784da-e67d-479c-bda3-4b5f5568bed6 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:07.184928 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f784da-e67d-479c-bda3-4b5f5568bed6 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:07.190854 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e58e1326-9b41-4894-97f0-78c6c94b032d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:07.191521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e1326-9b41-4894-97f0-78c6c94b032d admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:07.191801 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e1326-9b41-4894-97f0-78c6c94b032d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:07.192103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e58e1326-9b41-4894-97f0-78c6c94b032d admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:07.192831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e1326-9b41-4894-97f0-78c6c94b032d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:07.194950 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e58e1326-9b41-4894-97f0-78c6c94b032d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:07.205604 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1396/2780] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:18:07.209727 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5fcc2-06bd-46ac-9931-940859d21940 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:07.221356 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11e5fcc2-06bd-46ac-9931-940859d21940 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:07.226154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116f761-7cd9-4bd9-8885-6780eb722699 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:07.227090 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116f761-7cd9-4bd9-8885-6780eb722699 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:07.227405 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116f761-7cd9-4bd9-8885-6780eb722699 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:07.227687 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1116f761-7cd9-4bd9-8885-6780eb722699 admin admin] PATH_INFO: `/v3/projects/33cb8e59de0147faa8b5f218b5ad92a5/users/c45f68d55c4d4a8cabc7651099839f37/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:07.243561 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116f761-7cd9-4bd9-8885-6780eb722699 admin admin] RBAC: Authorizing `identity:create_grant(project_id=33cb8e59de0147faa8b5f218b5ad92a5, user_id=c45f68d55c4d4a8cabc7651099839f37, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:07.246329 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1116f761-7cd9-4bd9-8885-6780eb722699 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:07.249016 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:07.266134 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1385/2781] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:18:07 2022] PUT /identity/v3/projects/33cb8e59de0147faa8b5f218b5ad92a5/users/c45f68d55c4d4a8cabc7651099839f37/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:07.278706 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0762afd-1fb4-4c5a-af67-2db88f166127 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:07.279119 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0762afd-1fb4-4c5a-af67-2db88f166127 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:07.279273 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0762afd-1fb4-4c5a-af67-2db88f166127 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:07.311628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a0762afd-1fb4-4c5a-af67-2db88f166127 None None] MFA Rules not processed for user `c45f68d55c4d4a8cabc7651099839f37`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:07.312857 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a0762afd-1fb4-4c5a-af67-2db88f166127 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:07.378224 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1397/2782] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:07 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:07.383514 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116f761-7cd9-4bd9-8885-6780eb722699 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:07.445183 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1116f761-7cd9-4bd9-8885-6780eb722699 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:07.449782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06979531-907f-4db3-8777-b033182ed232 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:07.450468 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06979531-907f-4db3-8777-b033182ed232 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:07.450759 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06979531-907f-4db3-8777-b033182ed232 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:07.451042 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06979531-907f-4db3-8777-b033182ed232 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:07.451733 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06979531-907f-4db3-8777-b033182ed232 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:07.453618 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06979531-907f-4db3-8777-b033182ed232 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:07.478821 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1386/2783] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:18:07 2022] POST /identity/v3/projects => generated 389 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:18:07.482906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0762afd-1fb4-4c5a-af67-2db88f166127 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:07.493927 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0762afd-1fb4-4c5a-af67-2db88f166127 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:07.497514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:07.551497 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127e2e0-0af8-4805-b6fb-6be0e4b65702 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:07.552167 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127e2e0-0af8-4805-b6fb-6be0e4b65702 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:07.552453 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127e2e0-0af8-4805-b6fb-6be0e4b65702 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:07.552729 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9127e2e0-0af8-4805-b6fb-6be0e4b65702 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:07.553430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127e2e0-0af8-4805-b6fb-6be0e4b65702 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:07.555886 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9127e2e0-0af8-4805-b6fb-6be0e4b65702 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:07.578349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1398/2784] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:18:07 2022] POST /identity/v3/users => generated 301 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:18:07.582576 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06979531-907f-4db3-8777-b033182ed232 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:07.593121 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06979531-907f-4db3-8777-b033182ed232 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:07.597853 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af950c23-be22-4571-9c53-f08ff8b07f4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:07.598552 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af950c23-be22-4571-9c53-f08ff8b07f4e admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:07.598866 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af950c23-be22-4571-9c53-f08ff8b07f4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:07.599151 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af950c23-be22-4571-9c53-f08ff8b07f4e admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:07.599995 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af950c23-be22-4571-9c53-f08ff8b07f4e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:07.601933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af950c23-be22-4571-9c53-f08ff8b07f4e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:07.612957 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1387/2785] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:18:07.617707 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127e2e0-0af8-4805-b6fb-6be0e4b65702 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:07.627919 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9127e2e0-0af8-4805-b6fb-6be0e4b65702 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:07.633239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba828fe-fd2d-4e1b-865d-ef48b0857309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:07.634115 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba828fe-fd2d-4e1b-865d-ef48b0857309 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:07.634440 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba828fe-fd2d-4e1b-865d-ef48b0857309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:07.634723 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fba828fe-fd2d-4e1b-865d-ef48b0857309 admin admin] PATH_INFO: `/v3/projects/6e6cc37304734d0e965bdad010378798/users/d377f8a904994ac6834b9c7115f42774/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:07.649523 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba828fe-fd2d-4e1b-865d-ef48b0857309 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e6cc37304734d0e965bdad010378798, user_id=d377f8a904994ac6834b9c7115f42774, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:07.652218 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fba828fe-fd2d-4e1b-865d-ef48b0857309 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:07.665001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1399/2786] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:18:07 2022] PUT /identity/v3/projects/6e6cc37304734d0e965bdad010378798/users/d377f8a904994ac6834b9c7115f42774/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:18:07.668961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af950c23-be22-4571-9c53-f08ff8b07f4e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:07.679933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af950c23-be22-4571-9c53-f08ff8b07f4e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:07.736174 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-750b2617-7fde-456f-ad26-31a2818693dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:07.736791 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b2617-7fde-456f-ad26-31a2818693dc admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:07.737103 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b2617-7fde-456f-ad26-31a2818693dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:07.737377 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-750b2617-7fde-456f-ad26-31a2818693dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:07.738100 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b2617-7fde-456f-ad26-31a2818693dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:07.739717 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-750b2617-7fde-456f-ad26-31a2818693dc admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:07.750238 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1388/2787] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:07 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:18:07.754708 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba828fe-fd2d-4e1b-865d-ef48b0857309 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:07.765018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fba828fe-fd2d-4e1b-865d-ef48b0857309 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:07.769714 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b507b-2b54-4edf-9813-3e9f596a5e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:07.770630 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b507b-2b54-4edf-9813-3e9f596a5e18 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:07.770915 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b507b-2b54-4edf-9813-3e9f596a5e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:07.771193 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90b507b-2b54-4edf-9813-3e9f596a5e18 admin admin] PATH_INFO: `/v3/projects/6e6cc37304734d0e965bdad010378798/users/d377f8a904994ac6834b9c7115f42774/roles/6964639a7f6c41e8a6701ef533151c41` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:07.773198 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b507b-2b54-4edf-9813-3e9f596a5e18 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6e6cc37304734d0e965bdad010378798, user_id=d377f8a904994ac6834b9c7115f42774, role_id=6964639a7f6c41e8a6701ef533151c41)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:07.775887 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90b507b-2b54-4edf-9813-3e9f596a5e18 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:07.786133 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1400/2788] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:18:07 2022] PUT /identity/v3/projects/6e6cc37304734d0e965bdad010378798/users/d377f8a904994ac6834b9c7115f42774/roles/6964639a7f6c41e8a6701ef533151c41 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:07.796706 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6b238-19d7-4692-9e4c-3cc9fb2e1187 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:07.797124 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6b238-19d7-4692-9e4c-3cc9fb2e1187 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:07.797413 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c6b238-19d7-4692-9e4c-3cc9fb2e1187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:07.824122 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-b5c6b238-19d7-4692-9e4c-3cc9fb2e1187 None None] MFA Rules not processed for user `d377f8a904994ac6834b9c7115f42774`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:07.825339 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b5c6b238-19d7-4692-9e4c-3cc9fb2e1187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:07.887901 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:07.908230 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1389/2789] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:07 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:07.921578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b507b-2b54-4edf-9813-3e9f596a5e18 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:07.924102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a90b507b-2b54-4edf-9813-3e9f596a5e18 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:07.984881 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90b507b-2b54-4edf-9813-3e9f596a5e18 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:07.988965 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36127f7-b516-4bb1-a885-07b691d7827f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:07.989556 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36127f7-b516-4bb1-a885-07b691d7827f service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:07.989840 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36127f7-b516-4bb1-a885-07b691d7827f service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:07.990117 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36127f7-b516-4bb1-a885-07b691d7827f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:08.000006 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36127f7-b516-4bb1-a885-07b691d7827f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:08.001487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36127f7-b516-4bb1-a885-07b691d7827f service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:08.011525 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1401/2790] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:18:07 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:18:08.139366 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6b238-19d7-4692-9e4c-3cc9fb2e1187 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:08.205782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c6b238-19d7-4692-9e4c-3cc9fb2e1187 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:08.212788 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8b415-d5d8-427c-87ce-7bae32451041 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:08.214088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8b415-d5d8-427c-87ce-7bae32451041 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:08.214556 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8b415-d5d8-427c-87ce-7bae32451041 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:08.214973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ed8b415-d5d8-427c-87ce-7bae32451041 admin admin] PATH_INFO: `/v3/users/c45f68d55c4d4a8cabc7651099839f37` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:08.216712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8b415-d5d8-427c-87ce-7bae32451041 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c45f68d55c4d4a8cabc7651099839f37)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:08.219850 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ed8b415-d5d8-427c-87ce-7bae32451041 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:08.269683 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8ed8b415-d5d8-427c-87ce-7bae32451041 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c45f68d55c4d4a8cabc7651099839f37'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:08.270415 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8ed8b415-d5d8-427c-87ce-7bae32451041 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c45f68d55c4d4a8cabc7651099839f37'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:08.278764 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8ed8b415-d5d8-427c-87ce-7bae32451041 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c45f68d55c4d4a8cabc7651099839f37'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:08.279409 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8ed8b415-d5d8-427c-87ce-7bae32451041 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c45f68d55c4d4a8cabc7651099839f37'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:08.292132 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8ed8b415-d5d8-427c-87ce-7bae32451041 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c45f68d55c4d4a8cabc7651099839f37'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:08.304610 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1390/2791] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:18:08 2022] DELETE /identity/v3/users/c45f68d55c4d4a8cabc7651099839f37 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:18:08.308989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3fb96-17b7-468e-a709-c0abd560c376 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:08.309499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3fb96-17b7-468e-a709-c0abd560c376 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:08.309757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd3fb96-17b7-468e-a709-c0abd560c376 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:08.330816 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-dbd3fb96-17b7-468e-a709-c0abd560c376 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:08.332041 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dbd3fb96-17b7-468e-a709-c0abd560c376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:08.393719 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1402/2792] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:08.404062 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8b415-d5d8-427c-87ce-7bae32451041 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:08.405485 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8ed8b415-d5d8-427c-87ce-7bae32451041 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:08.464989 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ed8b415-d5d8-427c-87ce-7bae32451041 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:08.468709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b5620-1c22-4757-acb2-50eb52b0b952 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:08.469691 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b5620-1c22-4757-acb2-50eb52b0b952 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:08.469808 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b5620-1c22-4757-acb2-50eb52b0b952 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:08.470064 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f22b5620-1c22-4757-acb2-50eb52b0b952 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:08.480237 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b5620-1c22-4757-acb2-50eb52b0b952 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:08.481659 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f22b5620-1c22-4757-acb2-50eb52b0b952 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:08.491272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1391/2793] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:18:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:18:08.761035 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3fb96-17b7-468e-a709-c0abd560c376 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:08.762883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dbd3fb96-17b7-468e-a709-c0abd560c376 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:08.770166 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:08.778458 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd3fb96-17b7-468e-a709-c0abd560c376 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:08.784905 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04bfe8-b6f2-42c3-85a6-6df6c33471c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:08.785954 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04bfe8-b6f2-42c3-85a6-6df6c33471c1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:08.786241 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04bfe8-b6f2-42c3-85a6-6df6c33471c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:08.786562 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a04bfe8-b6f2-42c3-85a6-6df6c33471c1 admin admin] PATH_INFO: `/v3/projects/33cb8e59de0147faa8b5f218b5ad92a5` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:08.787808 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04bfe8-b6f2-42c3-85a6-6df6c33471c1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33cb8e59de0147faa8b5f218b5ad92a5)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:08.789953 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a04bfe8-b6f2-42c3-85a6-6df6c33471c1 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:08.837428 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1a04bfe8-b6f2-42c3-85a6-6df6c33471c1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33cb8e59de0147faa8b5f218b5ad92a5', 'request_id': 'req-1a04bfe8-b6f2-42c3-85a6-6df6c33471c1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:08.844807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1a04bfe8-b6f2-42c3-85a6-6df6c33471c1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33cb8e59de0147faa8b5f218b5ad92a5', 'request_id': 'req-1a04bfe8-b6f2-42c3-85a6-6df6c33471c1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:08.850628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1a04bfe8-b6f2-42c3-85a6-6df6c33471c1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33cb8e59de0147faa8b5f218b5ad92a5', 'request_id': 'req-1a04bfe8-b6f2-42c3-85a6-6df6c33471c1'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:08.862018 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1a04bfe8-b6f2-42c3-85a6-6df6c33471c1 admin admin] The token cache is being invalidate because project 33cb8e59de0147faa8b5f218b5ad92a5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:18:08.862392 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1a04bfe8-b6f2-42c3-85a6-6df6c33471c1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:08.863038 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1a04bfe8-b6f2-42c3-85a6-6df6c33471c1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:08.864591 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1403/2794] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:18:08 2022] DELETE /identity/v3/projects/33cb8e59de0147faa8b5f218b5ad92a5 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:08.869079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b5620-1c22-4757-acb2-50eb52b0b952 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:08.878739 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f22b5620-1c22-4757-acb2-50eb52b0b952 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:08.888595 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:08.943371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdd554f-ba22-411e-9294-a17aad4b68d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:08.944363 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd554f-ba22-411e-9294-a17aad4b68d9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:08.945265 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd554f-ba22-411e-9294-a17aad4b68d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:08.945630 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdd554f-ba22-411e-9294-a17aad4b68d9 admin admin] PATH_INFO: `/v3/users/d377f8a904994ac6834b9c7115f42774` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:08.947021 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd554f-ba22-411e-9294-a17aad4b68d9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d377f8a904994ac6834b9c7115f42774)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:08.949263 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdd554f-ba22-411e-9294-a17aad4b68d9 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:08.996829 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cfdd554f-ba22-411e-9294-a17aad4b68d9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd377f8a904994ac6834b9c7115f42774'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:08.997560 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cfdd554f-ba22-411e-9294-a17aad4b68d9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd377f8a904994ac6834b9c7115f42774'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:09.004701 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cfdd554f-ba22-411e-9294-a17aad4b68d9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd377f8a904994ac6834b9c7115f42774'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:09.005630 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cfdd554f-ba22-411e-9294-a17aad4b68d9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd377f8a904994ac6834b9c7115f42774'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:09.019773 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-cfdd554f-ba22-411e-9294-a17aad4b68d9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd377f8a904994ac6834b9c7115f42774'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:09.033205 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1392/2795] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:18:08 2022] DELETE /identity/v3/users/d377f8a904994ac6834b9c7115f42774 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:09.313449 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04bfe8-b6f2-42c3-85a6-6df6c33471c1 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:09.315129 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1a04bfe8-b6f2-42c3-85a6-6df6c33471c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:09.328476 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a04bfe8-b6f2-42c3-85a6-6df6c33471c1 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:09.392670 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d407551-a1de-4689-882a-d48a4ced4719 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:09.393550 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d407551-a1de-4689-882a-d48a4ced4719 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:09.393836 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d407551-a1de-4689-882a-d48a4ced4719 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:09.394112 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d407551-a1de-4689-882a-d48a4ced4719 admin admin] PATH_INFO: `/v3/projects/6e6cc37304734d0e965bdad010378798` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:09.395461 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d407551-a1de-4689-882a-d48a4ced4719 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6e6cc37304734d0e965bdad010378798)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:09.397397 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d407551-a1de-4689-882a-d48a4ced4719 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:09.445578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8d407551-a1de-4689-882a-d48a4ced4719 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6e6cc37304734d0e965bdad010378798', 'request_id': 'req-8d407551-a1de-4689-882a-d48a4ced4719'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:09.453542 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8d407551-a1de-4689-882a-d48a4ced4719 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6e6cc37304734d0e965bdad010378798', 'request_id': 'req-8d407551-a1de-4689-882a-d48a4ced4719'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:09.458945 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8d407551-a1de-4689-882a-d48a4ced4719 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6e6cc37304734d0e965bdad010378798', 'request_id': 'req-8d407551-a1de-4689-882a-d48a4ced4719'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:09.469879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8d407551-a1de-4689-882a-d48a4ced4719 admin admin] The token cache is being invalidate because project 6e6cc37304734d0e965bdad010378798 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:18:09.470164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8d407551-a1de-4689-882a-d48a4ced4719 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:09.470826 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-8d407551-a1de-4689-882a-d48a4ced4719 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:09.472367 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1404/2796] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:18:09 2022] DELETE /identity/v3/projects/6e6cc37304734d0e965bdad010378798 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:09.477907 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9df6b2-9a88-436f-8d6f-dcbac4db8c0e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:09.478442 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9df6b2-9a88-436f-8d6f-dcbac4db8c0e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:09.478548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf9df6b2-9a88-436f-8d6f-dcbac4db8c0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:09.498800 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-cf9df6b2-9a88-436f-8d6f-dcbac4db8c0e None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:09.500017 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-cf9df6b2-9a88-436f-8d6f-dcbac4db8c0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:09.560311 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1393/2797] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:09.568610 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24dcef-cef4-47db-ae4e-c70eecca7eea None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:09.569014 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24dcef-cef4-47db-ae4e-c70eecca7eea None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:09.569236 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a24dcef-cef4-47db-ae4e-c70eecca7eea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:09.588322 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-5a24dcef-cef4-47db-ae4e-c70eecca7eea None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:09.589523 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-5a24dcef-cef4-47db-ae4e-c70eecca7eea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:09.597459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1405/2798] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:09.602384 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9df6b2-9a88-436f-8d6f-dcbac4db8c0e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:09.614193 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf9df6b2-9a88-436f-8d6f-dcbac4db8c0e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:09.619563 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed6843-de32-436c-93c0-e38ac28c6d92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:09.620427 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed6843-de32-436c-93c0-e38ac28c6d92 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:09.620793 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed6843-de32-436c-93c0-e38ac28c6d92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:09.621069 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bed6843-de32-436c-93c0-e38ac28c6d92 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:09.621820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed6843-de32-436c-93c0-e38ac28c6d92 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:18:09.622236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed6843-de32-436c-93c0-e38ac28c6d92 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:09.623886 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bed6843-de32-436c-93c0-e38ac28c6d92 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:09.638450 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1394/2799] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:18:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:18:09.641551 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24dcef-cef4-47db-ae4e-c70eecca7eea None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:09.652720 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a24dcef-cef4-47db-ae4e-c70eecca7eea None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:09.657466 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24295691-a514-444b-8f75-1aedc92e0396 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:09.658119 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24295691-a514-444b-8f75-1aedc92e0396 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:09.658437 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24295691-a514-444b-8f75-1aedc92e0396 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:09.658719 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24295691-a514-444b-8f75-1aedc92e0396 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:09.659413 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24295691-a514-444b-8f75-1aedc92e0396 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:09.661315 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24295691-a514-444b-8f75-1aedc92e0396 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:09.666387 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:09.689576 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1406/2800] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:18:09 2022] POST /identity/v3/projects => generated 373 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:18:09.693492 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed6843-de32-436c-93c0-e38ac28c6d92 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:09.703415 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bed6843-de32-436c-93c0-e38ac28c6d92 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:09.716160 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:09.762272 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f343d3b-4225-48a9-ad0b-bc19442a94fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:09.762977 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f343d3b-4225-48a9-ad0b-bc19442a94fb admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:09.763265 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f343d3b-4225-48a9-ad0b-bc19442a94fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:09.763544 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f343d3b-4225-48a9-ad0b-bc19442a94fb admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:09.764236 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f343d3b-4225-48a9-ad0b-bc19442a94fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:09.766138 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f343d3b-4225-48a9-ad0b-bc19442a94fb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:09.786487 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1395/2801] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:18:09 2022] POST /identity/v3/users => generated 287 bytes in 93 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:18:09.793090 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24295691-a514-444b-8f75-1aedc92e0396 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:09.803644 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24295691-a514-444b-8f75-1aedc92e0396 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:09.808218 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f72bca-0ade-49e4-878e-12d2a81ceafe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:09.808883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f72bca-0ade-49e4-878e-12d2a81ceafe admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:09.809148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f72bca-0ade-49e4-878e-12d2a81ceafe admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:09.809426 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f72bca-0ade-49e4-878e-12d2a81ceafe admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:09.810179 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f72bca-0ade-49e4-878e-12d2a81ceafe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:09.811796 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f72bca-0ade-49e4-878e-12d2a81ceafe admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:09.821771 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1407/2802] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:18:09.825707 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f343d3b-4225-48a9-ad0b-bc19442a94fb admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:09.836551 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f343d3b-4225-48a9-ad0b-bc19442a94fb admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:09.841172 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69ae3620-8762-497f-bd90-77101ff2d6cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:09.841862 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae3620-8762-497f-bd90-77101ff2d6cb admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:09.842152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae3620-8762-497f-bd90-77101ff2d6cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:09.842476 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ae3620-8762-497f-bd90-77101ff2d6cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:09.843271 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae3620-8762-497f-bd90-77101ff2d6cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:09.844897 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69ae3620-8762-497f-bd90-77101ff2d6cb admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:09.855771 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1396/2803] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:09 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:18:09.860349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f72bca-0ade-49e4-878e-12d2a81ceafe admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:09.870430 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f72bca-0ade-49e4-878e-12d2a81ceafe admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:09.875254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066047de-b66b-4bc0-a2ae-a012d2b350c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:09.876134 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066047de-b66b-4bc0-a2ae-a012d2b350c9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:09.876424 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066047de-b66b-4bc0-a2ae-a012d2b350c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:09.876708 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-066047de-b66b-4bc0-a2ae-a012d2b350c9 admin admin] PATH_INFO: `/v3/projects/dd82df1de5344b2f8d59548547877207/users/e8687cb3369d4d68ae0503c541ffa92b/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:09.893257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066047de-b66b-4bc0-a2ae-a012d2b350c9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd82df1de5344b2f8d59548547877207, user_id=e8687cb3369d4d68ae0503c541ffa92b, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:09.895979 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-066047de-b66b-4bc0-a2ae-a012d2b350c9 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:09.912421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1408/2804] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:18:09 2022] PUT /identity/v3/projects/dd82df1de5344b2f8d59548547877207/users/e8687cb3369d4d68ae0503c541ffa92b/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:18:09.923403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2bd02-1488-45e3-b888-2fef16a0e524 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:09.923707 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2bd02-1488-45e3-b888-2fef16a0e524 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:09.924077 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a2bd02-1488-45e3-b888-2fef16a0e524 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:09.951288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-a8a2bd02-1488-45e3-b888-2fef16a0e524 None None] MFA Rules not processed for user `e8687cb3369d4d68ae0503c541ffa92b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:09.952518 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a8a2bd02-1488-45e3-b888-2fef16a0e524 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:10.018446 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1397/2805] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:09 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:10.030016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066047de-b66b-4bc0-a2ae-a012d2b350c9 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:10.031771 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-066047de-b66b-4bc0-a2ae-a012d2b350c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:10.089198 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-066047de-b66b-4bc0-a2ae-a012d2b350c9 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:10.092900 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c4c958-da89-4ad5-bf39-818910347257 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:10.093548 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4c958-da89-4ad5-bf39-818910347257 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:10.093836 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4c958-da89-4ad5-bf39-818910347257 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:10.094120 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c4c958-da89-4ad5-bf39-818910347257 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:10.102654 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4c958-da89-4ad5-bf39-818910347257 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:10.104091 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c4c958-da89-4ad5-bf39-818910347257 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:10.112895 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1409/2806] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:18:10 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:18:10.132572 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2bd02-1488-45e3-b888-2fef16a0e524 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:10.194998 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a2bd02-1488-45e3-b888-2fef16a0e524 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:10.199539 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9465510-2162-4b35-a8bd-f90028e74696 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:10.200404 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9465510-2162-4b35-a8bd-f90028e74696 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:10.200691 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9465510-2162-4b35-a8bd-f90028e74696 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:10.200973 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9465510-2162-4b35-a8bd-f90028e74696 admin admin] PATH_INFO: `/v3/users/e8687cb3369d4d68ae0503c541ffa92b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:10.202177 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9465510-2162-4b35-a8bd-f90028e74696 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e8687cb3369d4d68ae0503c541ffa92b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:10.204126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9465510-2162-4b35-a8bd-f90028e74696 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:10.232916 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:10.251748 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a9465510-2162-4b35-a8bd-f90028e74696 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e8687cb3369d4d68ae0503c541ffa92b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:10.252537 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a9465510-2162-4b35-a8bd-f90028e74696 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e8687cb3369d4d68ae0503c541ffa92b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:10.260698 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a9465510-2162-4b35-a8bd-f90028e74696 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e8687cb3369d4d68ae0503c541ffa92b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:10.261423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a9465510-2162-4b35-a8bd-f90028e74696 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e8687cb3369d4d68ae0503c541ffa92b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:10.275547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-a9465510-2162-4b35-a8bd-f90028e74696 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e8687cb3369d4d68ae0503c541ffa92b'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:10.288931 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1398/2807] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:18:10 2022] DELETE /identity/v3/users/e8687cb3369d4d68ae0503c541ffa92b => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:10.293708 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5314681-c91e-4f6f-b10d-80d12eb14428 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:10.294003 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5314681-c91e-4f6f-b10d-80d12eb14428 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:10.294285 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5314681-c91e-4f6f-b10d-80d12eb14428 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:10.312239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-c5314681-c91e-4f6f-b10d-80d12eb14428 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:10.313457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c5314681-c91e-4f6f-b10d-80d12eb14428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:10.381375 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1410/2808] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:10.388586 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9465510-2162-4b35-a8bd-f90028e74696 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:10.390098 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a9465510-2162-4b35-a8bd-f90028e74696 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:10.454246 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9465510-2162-4b35-a8bd-f90028e74696 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:10.460344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91431f6b-cd7d-417b-aac3-9e1ad42eae31 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:10.461129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91431f6b-cd7d-417b-aac3-9e1ad42eae31 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:10.461428 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91431f6b-cd7d-417b-aac3-9e1ad42eae31 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:10.461711 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91431f6b-cd7d-417b-aac3-9e1ad42eae31 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:10.474928 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91431f6b-cd7d-417b-aac3-9e1ad42eae31 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:10.476400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91431f6b-cd7d-417b-aac3-9e1ad42eae31 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:10.488006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1399/2809] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:18:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:18:10.829680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5314681-c91e-4f6f-b10d-80d12eb14428 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:10.831103 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-c5314681-c91e-4f6f-b10d-80d12eb14428 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:10.843833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5314681-c91e-4f6f-b10d-80d12eb14428 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:10.850962 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c4ade7e-9156-46db-b619-a651288b7a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:10.850962 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ade7e-9156-46db-b619-a651288b7a36 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:10.851309 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ade7e-9156-46db-b619-a651288b7a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:10.851418 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c4ade7e-9156-46db-b619-a651288b7a36 admin admin] PATH_INFO: `/v3/projects/dd82df1de5344b2f8d59548547877207` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:10.852604 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ade7e-9156-46db-b619-a651288b7a36 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd82df1de5344b2f8d59548547877207)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:10.854640 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c4ade7e-9156-46db-b619-a651288b7a36 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:10.910373 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1c4ade7e-9156-46db-b619-a651288b7a36 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd82df1de5344b2f8d59548547877207', 'request_id': 'req-1c4ade7e-9156-46db-b619-a651288b7a36'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:10.912282 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=94,keystone:INSERT=2,keystone:DELETE=4 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:10.919402 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1c4ade7e-9156-46db-b619-a651288b7a36 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd82df1de5344b2f8d59548547877207', 'request_id': 'req-1c4ade7e-9156-46db-b619-a651288b7a36'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:10.927627 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1c4ade7e-9156-46db-b619-a651288b7a36 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd82df1de5344b2f8d59548547877207', 'request_id': 'req-1c4ade7e-9156-46db-b619-a651288b7a36'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:10.939628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1c4ade7e-9156-46db-b619-a651288b7a36 admin admin] The token cache is being invalidate because project dd82df1de5344b2f8d59548547877207 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:18:10.939916 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1c4ade7e-9156-46db-b619-a651288b7a36 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:10.940561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-1c4ade7e-9156-46db-b619-a651288b7a36 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:10.942131 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1411/2810] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:18:10 2022] DELETE /identity/v3/projects/dd82df1de5344b2f8d59548547877207 => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:10.948106 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a095ae-5613-4bda-8b2f-849764f59fc5 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:10.948400 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a095ae-5613-4bda-8b2f-849764f59fc5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:10.948681 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99a095ae-5613-4bda-8b2f-849764f59fc5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:10.967699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-99a095ae-5613-4bda-8b2f-849764f59fc5 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:10.968959 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-99a095ae-5613-4bda-8b2f-849764f59fc5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:11.029434 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1400/2811] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:11.037025 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3bf48-b04f-4c59-bda2-2f32e59d009b None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:11.037314 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3bf48-b04f-4c59-bda2-2f32e59d009b None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:11.037593 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de3bf48-b04f-4c59-bda2-2f32e59d009b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:11.057898 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-7de3bf48-b04f-4c59-bda2-2f32e59d009b None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:11.059189 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7de3bf48-b04f-4c59-bda2-2f32e59d009b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:11.067310 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1412/2812] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:11.071602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a095ae-5613-4bda-8b2f-849764f59fc5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:11.081392 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99a095ae-5613-4bda-8b2f-849764f59fc5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:11.086536 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16bfaf-6db2-4598-98a5-9060119eb0cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:11.087202 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16bfaf-6db2-4598-98a5-9060119eb0cd admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:11.087493 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16bfaf-6db2-4598-98a5-9060119eb0cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:11.087778 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16bfaf-6db2-4598-98a5-9060119eb0cd admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:11.088548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16bfaf-6db2-4598-98a5-9060119eb0cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:18:11.088864 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16bfaf-6db2-4598-98a5-9060119eb0cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:11.090488 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16bfaf-6db2-4598-98a5-9060119eb0cd admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:11.102897 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1401/2813] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:18:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:18:11.107253 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3bf48-b04f-4c59-bda2-2f32e59d009b None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:11.116644 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de3bf48-b04f-4c59-bda2-2f32e59d009b None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:11.121317 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5affb42-d3b0-454a-9d70-e117dd53e297 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:11.121968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5affb42-d3b0-454a-9d70-e117dd53e297 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:11.122254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5affb42-d3b0-454a-9d70-e117dd53e297 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:11.122568 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5affb42-d3b0-454a-9d70-e117dd53e297 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:11.123264 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5affb42-d3b0-454a-9d70-e117dd53e297 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:11.125177 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5affb42-d3b0-454a-9d70-e117dd53e297 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:11.150833 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1413/2814] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:18:11 2022] POST /identity/v3/projects => generated 369 bytes in 44 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:18:11.155025 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16bfaf-6db2-4598-98a5-9060119eb0cd admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:11.165588 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16bfaf-6db2-4598-98a5-9060119eb0cd admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:11.198174 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:11.226218 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca11ff-2570-43cb-96bd-53223391dace admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:11.226877 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca11ff-2570-43cb-96bd-53223391dace admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:11.227163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca11ff-2570-43cb-96bd-53223391dace admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:11.227438 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acca11ff-2570-43cb-96bd-53223391dace admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:11.228144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca11ff-2570-43cb-96bd-53223391dace admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:11.230041 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acca11ff-2570-43cb-96bd-53223391dace admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:11.250136 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1402/2815] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:18:11 2022] POST /identity/v3/users => generated 285 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:18:11.254824 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5affb42-d3b0-454a-9d70-e117dd53e297 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:11.265142 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5affb42-d3b0-454a-9d70-e117dd53e297 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:11.269713 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d732eec-5247-460e-8c3f-759e56d8f93a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:11.270398 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d732eec-5247-460e-8c3f-759e56d8f93a admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:11.270688 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d732eec-5247-460e-8c3f-759e56d8f93a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:11.270964 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d732eec-5247-460e-8c3f-759e56d8f93a admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:11.271693 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d732eec-5247-460e-8c3f-759e56d8f93a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:11.273802 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d732eec-5247-460e-8c3f-759e56d8f93a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:11.284318 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1414/2816] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:18:11.288920 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca11ff-2570-43cb-96bd-53223391dace admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:11.298754 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acca11ff-2570-43cb-96bd-53223391dace admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:11.303516 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cebbf5b-8bb5-49c9-a900-5e4634b1937d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:11.304163 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebbf5b-8bb5-49c9-a900-5e4634b1937d admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:11.304446 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebbf5b-8bb5-49c9-a900-5e4634b1937d admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:11.304720 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cebbf5b-8bb5-49c9-a900-5e4634b1937d admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:11.305451 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebbf5b-8bb5-49c9-a900-5e4634b1937d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:11.307072 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4cebbf5b-8bb5-49c9-a900-5e4634b1937d admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:11.316866 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1403/2817] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:11 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:18:11.320866 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d732eec-5247-460e-8c3f-759e56d8f93a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:11.331108 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d732eec-5247-460e-8c3f-759e56d8f93a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:11.336263 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c66ceb-d2bf-4bf5-bcf5-f231673062c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:11.337131 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c66ceb-d2bf-4bf5-bcf5-f231673062c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:11.337416 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c66ceb-d2bf-4bf5-bcf5-f231673062c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:11.337724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90c66ceb-d2bf-4bf5-bcf5-f231673062c7 admin admin] PATH_INFO: `/v3/projects/ad6940a9c1494bb08ebeee095ab169e3/users/88c0ba8dd3c940fbbd6ebe1ca0ade310/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:11.352767 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c66ceb-d2bf-4bf5-bcf5-f231673062c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ad6940a9c1494bb08ebeee095ab169e3, user_id=88c0ba8dd3c940fbbd6ebe1ca0ade310, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:11.355471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90c66ceb-d2bf-4bf5-bcf5-f231673062c7 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:11.369015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1415/2818] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:18:11 2022] PUT /identity/v3/projects/ad6940a9c1494bb08ebeee095ab169e3/users/88c0ba8dd3c940fbbd6ebe1ca0ade310/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:11.380114 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d23309-8244-46ff-8916-4bf58378cc3e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:11.380431 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d23309-8244-46ff-8916-4bf58378cc3e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:11.380694 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d23309-8244-46ff-8916-4bf58378cc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:11.411397 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-53d23309-8244-46ff-8916-4bf58378cc3e None None] MFA Rules not processed for user `88c0ba8dd3c940fbbd6ebe1ca0ade310`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:11.412814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-53d23309-8244-46ff-8916-4bf58378cc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:11.481410 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1404/2819] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:11 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:11.493510 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c66ceb-d2bf-4bf5-bcf5-f231673062c7 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:11.495443 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-90c66ceb-d2bf-4bf5-bcf5-f231673062c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:11.574429 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90c66ceb-d2bf-4bf5-bcf5-f231673062c7 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:11.578141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3723c0c-7076-4683-bfe9-0f7f113f3f29 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f80d59fec4994e689e857ba2087d3a23', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:11.578824 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3723c0c-7076-4683-bfe9-0f7f113f3f29 service glance] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:11.579108 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3723c0c-7076-4683-bfe9-0f7f113f3f29 service glance] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:11.579388 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3723c0c-7076-4683-bfe9-0f7f113f3f29 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:11.589499 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3723c0c-7076-4683-bfe9-0f7f113f3f29 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:11.591108 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3723c0c-7076-4683-bfe9-0f7f113f3f29 service glance] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:11.601107 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1416/2820] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:18:11 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:18:21.478406 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=50,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:21.596845 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:INSERT=3,keystone:DELETE=3,keystone:SELECT=68 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:22.707785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf53da4-bc99-4695-b380-96a33c4aa255 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:22.708410 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf53da4-bc99-4695-b380-96a33c4aa255 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:22.708410 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adf53da4-bc99-4695-b380-96a33c4aa255 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:22.768360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-adf53da4-bc99-4695-b380-96a33c4aa255 None None] MFA Rules not processed for user `52076f59f68f4d77a0dadbf7a98d2b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:22.769577 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-adf53da4-bc99-4695-b380-96a33c4aa255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:22.843694 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1405/2821] 10.222.0.49 () {58 vars in 1009 bytes} [Mon Jul 11 14:18:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 137 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:22.894603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3723c0c-7076-4683-bfe9-0f7f113f3f29 service glance] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:22.896628 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-f3723c0c-7076-4683-bfe9-0f7f113f3f29 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:23.010257 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3723c0c-7076-4683-bfe9-0f7f113f3f29 service glance] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:23.015502 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee88ad5-26b9-4573-b914-4e95ca7ad696 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c1369033a44c1c81d4c1c9ff2bbedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:23.016153 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee88ad5-26b9-4573-b914-4e95ca7ad696 service swift] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:23.016481 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee88ad5-26b9-4573-b914-4e95ca7ad696 service swift] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:23.016758 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee88ad5-26b9-4573-b914-4e95ca7ad696 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:23.025877 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee88ad5-26b9-4573-b914-4e95ca7ad696 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:23.027565 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee88ad5-26b9-4573-b914-4e95ca7ad696 service swift] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:23.036371 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1417/2822] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 11 14:18:22 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 142 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 14:18:24.982053 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf53da4-bc99-4695-b380-96a33c4aa255 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:24.983547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-adf53da4-bc99-4695-b380-96a33c4aa255 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:25.072484 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adf53da4-bc99-4695-b380-96a33c4aa255 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:25.079515 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb422397-9ffe-48a4-be7f-4fdf5260da7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:25.080662 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb422397-9ffe-48a4-be7f-4fdf5260da7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:25.081024 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb422397-9ffe-48a4-be7f-4fdf5260da7a admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:25.081509 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb422397-9ffe-48a4-be7f-4fdf5260da7a admin admin] PATH_INFO: `/v3/users/fb98153d946f4e41a36c94a6dd19ce39` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:25.083922 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb422397-9ffe-48a4-be7f-4fdf5260da7a admin admin] RBAC: Authorizing `identity:delete_user(user_id=fb98153d946f4e41a36c94a6dd19ce39)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:25.087427 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb422397-9ffe-48a4-be7f-4fdf5260da7a admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:25.143593 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fb422397-9ffe-48a4-be7f-4fdf5260da7a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fb98153d946f4e41a36c94a6dd19ce39'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:25.144377 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fb422397-9ffe-48a4-be7f-4fdf5260da7a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fb98153d946f4e41a36c94a6dd19ce39'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:25.155974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fb422397-9ffe-48a4-be7f-4fdf5260da7a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fb98153d946f4e41a36c94a6dd19ce39'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:25.157188 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fb422397-9ffe-48a4-be7f-4fdf5260da7a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fb98153d946f4e41a36c94a6dd19ce39'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:25.178717 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-fb422397-9ffe-48a4-be7f-4fdf5260da7a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fb98153d946f4e41a36c94a6dd19ce39'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:25.182023 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:25.200456 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1406/2823] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:18:24 2022] DELETE /identity/v3/users/fb98153d946f4e41a36c94a6dd19ce39 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:18:25.205230 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6a3d7-c2ff-46e1-be01-94a8b9941d92 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:25.205738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6a3d7-c2ff-46e1-be01-94a8b9941d92 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:25.205937 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6a3d7-c2ff-46e1-be01-94a8b9941d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:25.224963 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a5c6a3d7-c2ff-46e1-be01-94a8b9941d92 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:25.226195 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a5c6a3d7-c2ff-46e1-be01-94a8b9941d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:25.315738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1418/2824] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:25.326436 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb422397-9ffe-48a4-be7f-4fdf5260da7a admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:25.328115 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-fb422397-9ffe-48a4-be7f-4fdf5260da7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:25.432053 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb422397-9ffe-48a4-be7f-4fdf5260da7a admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:25.436376 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac645974-51f4-44cc-94dd-7e4b9dcecec9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:25.437060 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac645974-51f4-44cc-94dd-7e4b9dcecec9 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:25.437361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac645974-51f4-44cc-94dd-7e4b9dcecec9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:25.437618 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac645974-51f4-44cc-94dd-7e4b9dcecec9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:25.453710 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac645974-51f4-44cc-94dd-7e4b9dcecec9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:25.455371 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac645974-51f4-44cc-94dd-7e4b9dcecec9 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:25.470046 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1407/2825] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:18:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:18:25.858854 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6a3d7-c2ff-46e1-be01-94a8b9941d92 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:25.861291 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a5c6a3d7-c2ff-46e1-be01-94a8b9941d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:25.881555 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6a3d7-c2ff-46e1-be01-94a8b9941d92 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:25.887384 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f473d-74e7-465c-8542-73525621b3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:25.888517 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f473d-74e7-465c-8542-73525621b3db admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:25.888517 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f473d-74e7-465c-8542-73525621b3db admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:25.888805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d82f473d-74e7-465c-8542-73525621b3db admin admin] PATH_INFO: `/v3/projects/4cdede8268f746daa1be618a34f54143` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:25.890170 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f473d-74e7-465c-8542-73525621b3db admin admin] RBAC: Authorizing `identity:delete_project(project_id=4cdede8268f746daa1be618a34f54143)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:25.892182 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d82f473d-74e7-465c-8542-73525621b3db admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:25.951985 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d82f473d-74e7-465c-8542-73525621b3db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4cdede8268f746daa1be618a34f54143', 'request_id': 'req-d82f473d-74e7-465c-8542-73525621b3db'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:25.962796 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d82f473d-74e7-465c-8542-73525621b3db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4cdede8268f746daa1be618a34f54143', 'request_id': 'req-d82f473d-74e7-465c-8542-73525621b3db'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:25.970921 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d82f473d-74e7-465c-8542-73525621b3db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4cdede8268f746daa1be618a34f54143', 'request_id': 'req-d82f473d-74e7-465c-8542-73525621b3db'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:25.972291 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:25.989781 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d82f473d-74e7-465c-8542-73525621b3db admin admin] The token cache is being invalidate because project 4cdede8268f746daa1be618a34f54143 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:18:25.990495 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d82f473d-74e7-465c-8542-73525621b3db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:25.991432 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-d82f473d-74e7-465c-8542-73525621b3db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:25.993744 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1419/2826] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:18:25 2022] DELETE /identity/v3/projects/4cdede8268f746daa1be618a34f54143 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:35.462993 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=30 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:35.988164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:43.067040 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416090fb-2006-4d5e-8457-93ae097bd42e None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:43.067580 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416090fb-2006-4d5e-8457-93ae097bd42e None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:43.067580 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-416090fb-2006-4d5e-8457-93ae097bd42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:43.086681 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-416090fb-2006-4d5e-8457-93ae097bd42e None None] MFA Rules not processed for user `52076f59f68f4d77a0dadbf7a98d2b57`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:43.087891 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-416090fb-2006-4d5e-8457-93ae097bd42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:43.143259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1408/2827] 10.222.0.49 () {58 vars in 967 bytes} [Mon Jul 11 14:18:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:43.156014 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f473d-74e7-465c-8542-73525621b3db admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:43.157492 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d82f473d-74e7-465c-8542-73525621b3db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:43.241668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d82f473d-74e7-465c-8542-73525621b3db admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:43.246746 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13baf110-2cb0-454b-9d99-f6f62c497cc7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'd9c1369033a44c1c81d4c1c9ff2bbedb', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:43.247414 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baf110-2cb0-454b-9d99-f6f62c497cc7 service swift] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:43.247704 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baf110-2cb0-454b-9d99-f6f62c497cc7 service swift] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:43.247985 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13baf110-2cb0-454b-9d99-f6f62c497cc7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:43.257644 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13baf110-2cb0-454b-9d99-f6f62c497cc7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:43.259514 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13baf110-2cb0-454b-9d99-f6f62c497cc7 service swift] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:43.268733 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1420/2828] 10.222.0.49 () {60 vars in 1384 bytes} [Mon Jul 11 14:18:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 11 14:18:45.178968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416090fb-2006-4d5e-8457-93ae097bd42e None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:45.180998 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-416090fb-2006-4d5e-8457-93ae097bd42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:45.255106 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13baf110-2cb0-454b-9d99-f6f62c497cc7 service swift] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:45.258164 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-13baf110-2cb0-454b-9d99-f6f62c497cc7 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:45.268977 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-416090fb-2006-4d5e-8457-93ae097bd42e None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:45.274369 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782516b4-543e-4558-aa43-b4c6109514ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee29b7f1cea64d6781e1df5c8021363f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:45.275102 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782516b4-543e-4558-aa43-b4c6109514ed service placement] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:45.275395 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782516b4-543e-4558-aa43-b4c6109514ed service placement] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:45.275680 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782516b4-543e-4558-aa43-b4c6109514ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:45.277603 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-782516b4-543e-4558-aa43-b4c6109514ed service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:45.290076 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782516b4-543e-4558-aa43-b4c6109514ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:45.291695 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782516b4-543e-4558-aa43-b4c6109514ed service placement] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:45.303550 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1409/2829] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:18:45 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:18:45.335803 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13baf110-2cb0-454b-9d99-f6f62c497cc7 service swift] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:45.340394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77770a65-4aee-4155-ba5f-ec39fb86b09d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:45.341299 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77770a65-4aee-4155-ba5f-ec39fb86b09d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:45.341587 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77770a65-4aee-4155-ba5f-ec39fb86b09d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:45.341894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77770a65-4aee-4155-ba5f-ec39fb86b09d admin admin] PATH_INFO: `/v3/users/9ae9e78d25e94319801ce906327ada27` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:45.343120 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77770a65-4aee-4155-ba5f-ec39fb86b09d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ae9e78d25e94319801ce906327ada27)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:45.345072 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77770a65-4aee-4155-ba5f-ec39fb86b09d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:45.391841 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-77770a65-4aee-4155-ba5f-ec39fb86b09d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ae9e78d25e94319801ce906327ada27'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:45.403879 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-77770a65-4aee-4155-ba5f-ec39fb86b09d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ae9e78d25e94319801ce906327ada27'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:45.410857 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-77770a65-4aee-4155-ba5f-ec39fb86b09d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ae9e78d25e94319801ce906327ada27'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:45.411469 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-77770a65-4aee-4155-ba5f-ec39fb86b09d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ae9e78d25e94319801ce906327ada27'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:45.418743 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-77770a65-4aee-4155-ba5f-ec39fb86b09d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ae9e78d25e94319801ce906327ada27'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:45.426480 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1421/2830] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:18:45 2022] DELETE /identity/v3/users/9ae9e78d25e94319801ce906327ada27 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:45.594824 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782516b4-543e-4558-aa43-b4c6109514ed service placement] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:45.606977 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782516b4-543e-4558-aa43-b4c6109514ed service placement] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:45.670177 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a03b4f6-2ff9-4c97-9b4d-3861f5b97e0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:45.671078 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03b4f6-2ff9-4c97-9b4d-3861f5b97e0e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:45.671366 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03b4f6-2ff9-4c97-9b4d-3861f5b97e0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:45.671650 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a03b4f6-2ff9-4c97-9b4d-3861f5b97e0e admin admin] PATH_INFO: `/v3/projects/9a7d950a8bf049dcb5600fc64b3b6cc0` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:45.672843 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a03b4f6-2ff9-4c97-9b4d-3861f5b97e0e admin admin] RBAC: Authorizing `identity:delete_project(project_id=9a7d950a8bf049dcb5600fc64b3b6cc0)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:45.675078 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a03b4f6-2ff9-4c97-9b4d-3861f5b97e0e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:45.712900 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:45.722243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3a03b4f6-2ff9-4c97-9b4d-3861f5b97e0e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9a7d950a8bf049dcb5600fc64b3b6cc0', 'request_id': 'req-3a03b4f6-2ff9-4c97-9b4d-3861f5b97e0e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:45.728333 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3a03b4f6-2ff9-4c97-9b4d-3861f5b97e0e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9a7d950a8bf049dcb5600fc64b3b6cc0', 'request_id': 'req-3a03b4f6-2ff9-4c97-9b4d-3861f5b97e0e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:45.734897 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3a03b4f6-2ff9-4c97-9b4d-3861f5b97e0e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9a7d950a8bf049dcb5600fc64b3b6cc0', 'request_id': 'req-3a03b4f6-2ff9-4c97-9b4d-3861f5b97e0e'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:45.745985 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3a03b4f6-2ff9-4c97-9b4d-3861f5b97e0e admin admin] The token cache is being invalidate because project 9a7d950a8bf049dcb5600fc64b3b6cc0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:18:45.746273 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3a03b4f6-2ff9-4c97-9b4d-3861f5b97e0e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:45.746927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3a03b4f6-2ff9-4c97-9b4d-3861f5b97e0e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:45.748479 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1410/2831] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:18:45 2022] DELETE /identity/v3/projects/9a7d950a8bf049dcb5600fc64b3b6cc0 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:45.753508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d1f0b-0167-402b-ac9d-306c01add10f None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:45.753808 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d1f0b-0167-402b-ac9d-306c01add10f None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:45.754118 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a38d1f0b-0167-402b-ac9d-306c01add10f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:45.775895 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a38d1f0b-0167-402b-ac9d-306c01add10f None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:45.778095 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a38d1f0b-0167-402b-ac9d-306c01add10f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:45.784258 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:45.843459 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1422/2832] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:45.851670 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c96b1-e20d-42ca-897a-a8f6acf8515d None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:45.851960 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c96b1-e20d-42ca-897a-a8f6acf8515d None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:45.852240 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5c96b1-e20d-42ca-897a-a8f6acf8515d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:45.870729 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ba5c96b1-e20d-42ca-897a-a8f6acf8515d None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:45.871956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ba5c96b1-e20d-42ca-897a-a8f6acf8515d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:45.884007 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1411/2833] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:45.885569 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d1f0b-0167-402b-ac9d-306c01add10f None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:45.896965 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a38d1f0b-0167-402b-ac9d-306c01add10f None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:45.903655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab41ce-14ee-4fae-866c-9ce7f0d70f95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:45.904490 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab41ce-14ee-4fae-866c-9ce7f0d70f95 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:45.904892 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab41ce-14ee-4fae-866c-9ce7f0d70f95 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:45.905394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8eab41ce-14ee-4fae-866c-9ce7f0d70f95 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:45.906260 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab41ce-14ee-4fae-866c-9ce7f0d70f95 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:18:45.906705 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab41ce-14ee-4fae-866c-9ce7f0d70f95 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:45.908505 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8eab41ce-14ee-4fae-866c-9ce7f0d70f95 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:45.921992 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1423/2834] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:18:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:18:45.926740 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c96b1-e20d-42ca-897a-a8f6acf8515d None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:45.936809 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5c96b1-e20d-42ca-897a-a8f6acf8515d None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:45.941929 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e9c4b-985d-4e02-a860-57e138fd618f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:45.942679 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e9c4b-985d-4e02-a860-57e138fd618f admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:45.942967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e9c4b-985d-4e02-a860-57e138fd618f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:45.943248 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e97e9c4b-985d-4e02-a860-57e138fd618f admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:45.943977 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e9c4b-985d-4e02-a860-57e138fd618f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:45.946558 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e97e9c4b-985d-4e02-a860-57e138fd618f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:45.974840 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1412/2835] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:18:45 2022] POST /identity/v3/projects => generated 371 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:18:45.979181 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab41ce-14ee-4fae-866c-9ce7f0d70f95 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:45.989478 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8eab41ce-14ee-4fae-866c-9ce7f0d70f95 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:46.053894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ac84f-6a74-4054-91cf-3c5a8abb6d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:46.054843 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ac84f-6a74-4054-91cf-3c5a8abb6d3b admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:46.055131 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ac84f-6a74-4054-91cf-3c5a8abb6d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:46.055409 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15ac84f-6a74-4054-91cf-3c5a8abb6d3b admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:46.056160 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ac84f-6a74-4054-91cf-3c5a8abb6d3b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:46.058209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15ac84f-6a74-4054-91cf-3c5a8abb6d3b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:46.087652 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1424/2836] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:18:45 2022] POST /identity/v3/users => generated 286 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:18:46.092146 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e9c4b-985d-4e02-a860-57e138fd618f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:46.105982 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e97e9c4b-985d-4e02-a860-57e138fd618f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:46.110263 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa68ea-768c-4724-ad78-3e93bf0cb8d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:46.110964 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa68ea-768c-4724-ad78-3e93bf0cb8d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:46.111249 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa68ea-768c-4724-ad78-3e93bf0cb8d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:46.111530 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fa68ea-768c-4724-ad78-3e93bf0cb8d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:46.112276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa68ea-768c-4724-ad78-3e93bf0cb8d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:46.113881 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fa68ea-768c-4724-ad78-3e93bf0cb8d2 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:46.126453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1413/2837] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:18:46.130295 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ac84f-6a74-4054-91cf-3c5a8abb6d3b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:46.142355 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15ac84f-6a74-4054-91cf-3c5a8abb6d3b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:46.147435 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a41a5705-c0bc-4d01-ba93-87f30a885122 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:46.148839 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a5705-c0bc-4d01-ba93-87f30a885122 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:46.149129 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a5705-c0bc-4d01-ba93-87f30a885122 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:46.149405 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a41a5705-c0bc-4d01-ba93-87f30a885122 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:46.150148 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a5705-c0bc-4d01-ba93-87f30a885122 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:46.151805 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a41a5705-c0bc-4d01-ba93-87f30a885122 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:46.167073 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1425/2838] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:46 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:18:46.171627 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa68ea-768c-4724-ad78-3e93bf0cb8d2 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:46.183782 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fa68ea-768c-4724-ad78-3e93bf0cb8d2 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:46.191055 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d846454-0ab7-447b-b386-725d764aa042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:46.192038 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d846454-0ab7-447b-b386-725d764aa042 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:46.192341 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d846454-0ab7-447b-b386-725d764aa042 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:46.192712 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d846454-0ab7-447b-b386-725d764aa042 admin admin] PATH_INFO: `/v3/projects/78651f6706904f2a97278522911a253b/users/c60d354ece1f46049b3dbc7e5152ddd9/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:46.210459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d846454-0ab7-447b-b386-725d764aa042 admin admin] RBAC: Authorizing `identity:create_grant(project_id=78651f6706904f2a97278522911a253b, user_id=c60d354ece1f46049b3dbc7e5152ddd9, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:46.213754 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d846454-0ab7-447b-b386-725d764aa042 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:46.230010 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1414/2839] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:18:46 2022] PUT /identity/v3/projects/78651f6706904f2a97278522911a253b/users/c60d354ece1f46049b3dbc7e5152ddd9/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:46.240111 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06664fe9-980f-4435-8800-3cf95d6e092c None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:46.240482 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06664fe9-980f-4435-8800-3cf95d6e092c None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:46.240894 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06664fe9-980f-4435-8800-3cf95d6e092c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:46.273372 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-06664fe9-980f-4435-8800-3cf95d6e092c None None] MFA Rules not processed for user `c60d354ece1f46049b3dbc7e5152ddd9`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:46.274688 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-06664fe9-980f-4435-8800-3cf95d6e092c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:46.349254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1426/2840] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:46 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:46.361180 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d846454-0ab7-447b-b386-725d764aa042 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:46.363085 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-9d846454-0ab7-447b-b386-725d764aa042 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:46.426682 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d846454-0ab7-447b-b386-725d764aa042 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:46.431060 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0c48a8f-5fd0-474f-90aa-ac427727074d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:46.431731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c48a8f-5fd0-474f-90aa-ac427727074d service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:46.432020 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c48a8f-5fd0-474f-90aa-ac427727074d service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:46.432304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0c48a8f-5fd0-474f-90aa-ac427727074d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:46.441880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c48a8f-5fd0-474f-90aa-ac427727074d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:46.443372 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0c48a8f-5fd0-474f-90aa-ac427727074d service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:46.453974 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1415/2841] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:18:46 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:18:46.478013 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06664fe9-980f-4435-8800-3cf95d6e092c None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:46.483680 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:46.546353 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06664fe9-980f-4435-8800-3cf95d6e092c None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:46.551454 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6b1cd-5f15-49ee-a340-11424f672c5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:46.552361 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6b1cd-5f15-49ee-a340-11424f672c5d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:46.552647 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6b1cd-5f15-49ee-a340-11424f672c5d admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:46.552992 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa6b1cd-5f15-49ee-a340-11424f672c5d admin admin] PATH_INFO: `/v3/users/c60d354ece1f46049b3dbc7e5152ddd9` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:46.554819 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6b1cd-5f15-49ee-a340-11424f672c5d admin admin] RBAC: Authorizing `identity:delete_user(user_id=c60d354ece1f46049b3dbc7e5152ddd9)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:46.556900 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa6b1cd-5f15-49ee-a340-11424f672c5d admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:46.612407 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dfa6b1cd-5f15-49ee-a340-11424f672c5d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c60d354ece1f46049b3dbc7e5152ddd9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:46.625052 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dfa6b1cd-5f15-49ee-a340-11424f672c5d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c60d354ece1f46049b3dbc7e5152ddd9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:46.632859 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dfa6b1cd-5f15-49ee-a340-11424f672c5d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c60d354ece1f46049b3dbc7e5152ddd9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:46.633585 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dfa6b1cd-5f15-49ee-a340-11424f672c5d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c60d354ece1f46049b3dbc7e5152ddd9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:46.640714 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-dfa6b1cd-5f15-49ee-a340-11424f672c5d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c60d354ece1f46049b3dbc7e5152ddd9'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:46.647656 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1427/2842] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:18:46 2022] DELETE /identity/v3/users/c60d354ece1f46049b3dbc7e5152ddd9 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:46.652304 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd507c-fca9-4edd-be4d-9c317119ae03 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:46.652595 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd507c-fca9-4edd-be4d-9c317119ae03 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:46.652872 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26fd507c-fca9-4edd-be4d-9c317119ae03 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:46.671211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-26fd507c-fca9-4edd-be4d-9c317119ae03 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:46.672429 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-26fd507c-fca9-4edd-be4d-9c317119ae03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:46.733027 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1416/2843] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:46.743486 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6b1cd-5f15-49ee-a340-11424f672c5d admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:46.744966 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-dfa6b1cd-5f15-49ee-a340-11424f672c5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:46.805791 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa6b1cd-5f15-49ee-a340-11424f672c5d admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:46.809450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d3f1331-dea1-49ce-af92-691f20efef21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:46.810102 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f1331-dea1-49ce-af92-691f20efef21 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:46.810421 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f1331-dea1-49ce-af92-691f20efef21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:46.810700 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d3f1331-dea1-49ce-af92-691f20efef21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:46.819996 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f1331-dea1-49ce-af92-691f20efef21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:46.821419 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d3f1331-dea1-49ce-af92-691f20efef21 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:46.830825 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1428/2844] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:18:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:18:47.099121 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd507c-fca9-4edd-be4d-9c317119ae03 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:47.101596 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-26fd507c-fca9-4edd-be4d-9c317119ae03 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:47.110151 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:47.114033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26fd507c-fca9-4edd-be4d-9c317119ae03 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:47.120088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c68dc6-9d40-488a-8420-371337b56419 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:47.121043 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c68dc6-9d40-488a-8420-371337b56419 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:47.121407 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c68dc6-9d40-488a-8420-371337b56419 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:47.122126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c68dc6-9d40-488a-8420-371337b56419 admin admin] PATH_INFO: `/v3/projects/78651f6706904f2a97278522911a253b` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:47.123650 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c68dc6-9d40-488a-8420-371337b56419 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78651f6706904f2a97278522911a253b)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:47.126256 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c68dc6-9d40-488a-8420-371337b56419 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:47.173586 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-38c68dc6-9d40-488a-8420-371337b56419 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78651f6706904f2a97278522911a253b', 'request_id': 'req-38c68dc6-9d40-488a-8420-371337b56419'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:47.179331 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-38c68dc6-9d40-488a-8420-371337b56419 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78651f6706904f2a97278522911a253b', 'request_id': 'req-38c68dc6-9d40-488a-8420-371337b56419'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:47.186594 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-38c68dc6-9d40-488a-8420-371337b56419 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78651f6706904f2a97278522911a253b', 'request_id': 'req-38c68dc6-9d40-488a-8420-371337b56419'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:47.198018 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-38c68dc6-9d40-488a-8420-371337b56419 admin admin] The token cache is being invalidate because project 78651f6706904f2a97278522911a253b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:18:47.198334 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-38c68dc6-9d40-488a-8420-371337b56419 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:47.198955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-38c68dc6-9d40-488a-8420-371337b56419 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:18:47.200554 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1417/2845] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:18:47 2022] DELETE /identity/v3/projects/78651f6706904f2a97278522911a253b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:47.205472 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836d7bd-5221-4e57-a31d-11f8f2abb0de None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:47.205755 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836d7bd-5221-4e57-a31d-11f8f2abb0de None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:47.205933 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e836d7bd-5221-4e57-a31d-11f8f2abb0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:47.224151 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e836d7bd-5221-4e57-a31d-11f8f2abb0de None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:47.225367 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e836d7bd-5221-4e57-a31d-11f8f2abb0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:47.245846 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:47.291093 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1429/2846] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:47.296292 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b215534-9e35-4c72-827f-6fe7825276eb None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:47.296584 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b215534-9e35-4c72-827f-6fe7825276eb None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:47.296870 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b215534-9e35-4c72-827f-6fe7825276eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:47.315547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1b215534-9e35-4c72-827f-6fe7825276eb None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:47.316764 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1b215534-9e35-4c72-827f-6fe7825276eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:47.324953 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1418/2847] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:47.330024 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836d7bd-5221-4e57-a31d-11f8f2abb0de None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:47.340500 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e836d7bd-5221-4e57-a31d-11f8f2abb0de None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:47.345015 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c9d9a-06b5-4adb-9028-a84014f6ef03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:47.345658 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c9d9a-06b5-4adb-9028-a84014f6ef03 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:47.345945 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c9d9a-06b5-4adb-9028-a84014f6ef03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:47.346214 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970c9d9a-06b5-4adb-9028-a84014f6ef03 admin admin] PATH_INFO: `/v3/domains` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:47.346982 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c9d9a-06b5-4adb-9028-a84014f6ef03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80247) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:18:47.347311 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c9d9a-06b5-4adb-9028-a84014f6ef03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:47.348883 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970c9d9a-06b5-4adb-9028-a84014f6ef03 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:47.361123 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1430/2848] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:18:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:18:47.365344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b215534-9e35-4c72-827f-6fe7825276eb None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:47.374882 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b215534-9e35-4c72-827f-6fe7825276eb None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:47.379641 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38749234-f5c9-48a5-8ead-7e85ec9aacd0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:47.380389 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38749234-f5c9-48a5-8ead-7e85ec9aacd0 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:47.380683 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38749234-f5c9-48a5-8ead-7e85ec9aacd0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:47.380967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38749234-f5c9-48a5-8ead-7e85ec9aacd0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:47.381679 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38749234-f5c9-48a5-8ead-7e85ec9aacd0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:47.383681 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38749234-f5c9-48a5-8ead-7e85ec9aacd0 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:47.410548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1419/2849] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:18:47 2022] POST /identity/v3/projects => generated 383 bytes in 46 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 11 14:18:47.415313 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c9d9a-06b5-4adb-9028-a84014f6ef03 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:47.425378 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970c9d9a-06b5-4adb-9028-a84014f6ef03 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:47.482172 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296be466-e450-437d-a65b-b86a690c6398 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:47.482882 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296be466-e450-437d-a65b-b86a690c6398 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:47.483173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296be466-e450-437d-a65b-b86a690c6398 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:47.483457 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-296be466-e450-437d-a65b-b86a690c6398 admin admin] PATH_INFO: `/v3/users` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:47.484159 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296be466-e450-437d-a65b-b86a690c6398 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:47.486092 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-296be466-e450-437d-a65b-b86a690c6398 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:47.506084 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1431/2850] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:18:47 2022] POST /identity/v3/users => generated 292 bytes in 91 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:18:47.510697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38749234-f5c9-48a5-8ead-7e85ec9aacd0 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:47.521013 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38749234-f5c9-48a5-8ead-7e85ec9aacd0 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:47.526470 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72d01a-b4d9-49c1-b8bb-df92abd33104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:47.527137 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72d01a-b4d9-49c1-b8bb-df92abd33104 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:47.527427 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72d01a-b4d9-49c1-b8bb-df92abd33104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:47.527710 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d72d01a-b4d9-49c1-b8bb-df92abd33104 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:47.528452 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72d01a-b4d9-49c1-b8bb-df92abd33104 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:47.530258 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d72d01a-b4d9-49c1-b8bb-df92abd33104 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:47.541233 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1420/2851] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:18:47.545159 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296be466-e450-437d-a65b-b86a690c6398 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:47.555442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-296be466-e450-437d-a65b-b86a690c6398 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:47.560087 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd55aea-ff17-4241-af42-b4bd889d8f7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:47.560742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd55aea-ff17-4241-af42-b4bd889d8f7c admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:47.561029 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd55aea-ff17-4241-af42-b4bd889d8f7c admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:47.561305 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd55aea-ff17-4241-af42-b4bd889d8f7c admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:47.562030 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd55aea-ff17-4241-af42-b4bd889d8f7c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:47.563658 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd55aea-ff17-4241-af42-b4bd889d8f7c admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:47.575989 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1432/2852] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:18:47 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 11 14:18:47.580074 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72d01a-b4d9-49c1-b8bb-df92abd33104 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:47.591193 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d72d01a-b4d9-49c1-b8bb-df92abd33104 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:47.595984 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156786b9-e667-43f2-94c1-d16a8c9566b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:47.596878 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156786b9-e667-43f2-94c1-d16a8c9566b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:47.597164 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156786b9-e667-43f2-94c1-d16a8c9566b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:47.597453 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156786b9-e667-43f2-94c1-d16a8c9566b3 admin admin] PATH_INFO: `/v3/projects/58346265e4c244dcb15cd6ec2817773a/users/72a0be934143483e8cdcc25fb7b3c915/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:47.613954 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156786b9-e667-43f2-94c1-d16a8c9566b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=58346265e4c244dcb15cd6ec2817773a, user_id=72a0be934143483e8cdcc25fb7b3c915, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:47.616730 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156786b9-e667-43f2-94c1-d16a8c9566b3 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:47.641873 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1421/2853] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:18:47 2022] PUT /identity/v3/projects/58346265e4c244dcb15cd6ec2817773a/users/72a0be934143483e8cdcc25fb7b3c915/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 62 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:18:47.647446 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7e682-c408-4f13-adbd-26c98db85426 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:47.648470 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7e682-c408-4f13-adbd-26c98db85426 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:47.648752 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66b7e682-c408-4f13-adbd-26c98db85426 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:47.668312 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-66b7e682-c408-4f13-adbd-26c98db85426 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:47.669578 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-66b7e682-c408-4f13-adbd-26c98db85426 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:47.734380 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1433/2854] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:47.744350 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156786b9-e667-43f2-94c1-d16a8c9566b3 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:47.746267 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-156786b9-e667-43f2-94c1-d16a8c9566b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:47.807183 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156786b9-e667-43f2-94c1-d16a8c9566b3 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:47.810795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683f6db-cd89-4ce3-a2e2-b737be31f191 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:47.811448 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683f6db-cd89-4ce3-a2e2-b737be31f191 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:47.811735 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683f6db-cd89-4ce3-a2e2-b737be31f191 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:47.812013 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b683f6db-cd89-4ce3-a2e2-b737be31f191 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:47.822033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683f6db-cd89-4ce3-a2e2-b737be31f191 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:47.823513 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b683f6db-cd89-4ce3-a2e2-b737be31f191 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:47.833495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1422/2855] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:18:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:18:48.536276 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f107800-985b-4f66-8993-545cb116dab8 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:48.536846 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f107800-985b-4f66-8993-545cb116dab8 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:48.536846 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f107800-985b-4f66-8993-545cb116dab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:48.567169 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-1f107800-985b-4f66-8993-545cb116dab8 None None] MFA Rules not processed for user `72a0be934143483e8cdcc25fb7b3c915`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:48.568384 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1f107800-985b-4f66-8993-545cb116dab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:48.590634 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:48.640061 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1434/2856] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:48.651651 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683f6db-cd89-4ce3-a2e2-b737be31f191 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:18:48.653226 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-b683f6db-cd89-4ce3-a2e2-b737be31f191 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:48.686820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:18:48.715157 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b683f6db-cd89-4ce3-a2e2-b737be31f191 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:18:48.718754 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c2156-90e4-4c72-9bbc-09cca920755a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:18:48.719403 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c2156-90e4-4c72-9bbc-09cca920755a service cinder] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:48.719690 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c2156-90e4-4c72-9bbc-09cca920755a service cinder] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:48.719968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14c2156-90e4-4c72-9bbc-09cca920755a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:48.729083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c2156-90e4-4c72-9bbc-09cca920755a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:18:48.730576 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a14c2156-90e4-4c72-9bbc-09cca920755a service cinder] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:18:48.739680 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1423/2857] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:18:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:18:54.293137 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb15102-fb53-4914-92d0-3092b7771263 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:18:54.293693 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb15102-fb53-4914-92d0-3092b7771263 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:18:54.293792 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fb15102-fb53-4914-92d0-3092b7771263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:18:54.313531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-7fb15102-fb53-4914-92d0-3092b7771263 None None] MFA Rules not processed for user `72a0be934143483e8cdcc25fb7b3c915`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:18:54.314806 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7fb15102-fb53-4914-92d0-3092b7771263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:18:54.322035 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1435/2858] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:18:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:18:58.735491 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=14 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:19:04.310187 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=20 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:19:09.337829 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c2156-90e4-4c72-9bbc-09cca920755a service cinder] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:19:09.339536 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-a14c2156-90e4-4c72-9bbc-09cca920755a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:19:09.354209 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14c2156-90e4-4c72-9bbc-09cca920755a service cinder] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:19:09.359846 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4594dd-b85d-48f0-91f8-0fa937d0e744 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:19:09.360716 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4594dd-b85d-48f0-91f8-0fa937d0e744 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:19:09.361003 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4594dd-b85d-48f0-91f8-0fa937d0e744 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:19:09.361283 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4594dd-b85d-48f0-91f8-0fa937d0e744 admin admin] PATH_INFO: `/v3/users/88c0ba8dd3c940fbbd6ebe1ca0ade310` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:19:09.362537 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4594dd-b85d-48f0-91f8-0fa937d0e744 admin admin] RBAC: Authorizing `identity:delete_user(user_id=88c0ba8dd3c940fbbd6ebe1ca0ade310)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:19:09.364452 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4594dd-b85d-48f0-91f8-0fa937d0e744 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:19:09.412773 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0d4594dd-b85d-48f0-91f8-0fa937d0e744 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '88c0ba8dd3c940fbbd6ebe1ca0ade310'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:19:09.413458 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0d4594dd-b85d-48f0-91f8-0fa937d0e744 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '88c0ba8dd3c940fbbd6ebe1ca0ade310'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:19:09.420462 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0d4594dd-b85d-48f0-91f8-0fa937d0e744 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '88c0ba8dd3c940fbbd6ebe1ca0ade310'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:19:09.421154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0d4594dd-b85d-48f0-91f8-0fa937d0e744 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '88c0ba8dd3c940fbbd6ebe1ca0ade310'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:19:09.433288 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0d4594dd-b85d-48f0-91f8-0fa937d0e744 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '88c0ba8dd3c940fbbd6ebe1ca0ade310'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:19:09.445699 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1424/2859] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:19:09 2022] DELETE /identity/v3/users/88c0ba8dd3c940fbbd6ebe1ca0ade310 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:19:09.450281 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68d9c4-8db4-4cb2-839d-26eaf5603b62 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:19:09.450788 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68d9c4-8db4-4cb2-839d-26eaf5603b62 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:19:09.450908 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be68d9c4-8db4-4cb2-839d-26eaf5603b62 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:19:09.468504 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-be68d9c4-8db4-4cb2-839d-26eaf5603b62 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:19:09.469695 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-be68d9c4-8db4-4cb2-839d-26eaf5603b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:19:09.529082 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1436/2860] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:19:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:19:09.538967 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4594dd-b85d-48f0-91f8-0fa937d0e744 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:19:09.540451 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0d4594dd-b85d-48f0-91f8-0fa937d0e744 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:19:09.598077 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4594dd-b85d-48f0-91f8-0fa937d0e744 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:19:09.601736 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01364997-25f7-4d45-a498-13515ae3311c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:19:09.602423 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01364997-25f7-4d45-a498-13515ae3311c service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:19:09.602715 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01364997-25f7-4d45-a498-13515ae3311c service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:19:09.602996 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01364997-25f7-4d45-a498-13515ae3311c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:19:09.612154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01364997-25f7-4d45-a498-13515ae3311c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:19:09.613586 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01364997-25f7-4d45-a498-13515ae3311c service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:19:09.623597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1425/2861] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:19:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:19:09.895487 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68d9c4-8db4-4cb2-839d-26eaf5603b62 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:19:09.897009 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-be68d9c4-8db4-4cb2-839d-26eaf5603b62 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:19:09.911229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be68d9c4-8db4-4cb2-839d-26eaf5603b62 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:19:09.916778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43956ef2-a54d-4814-a66b-f1e648162b37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:19:09.917655 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43956ef2-a54d-4814-a66b-f1e648162b37 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:19:09.917946 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43956ef2-a54d-4814-a66b-f1e648162b37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:19:09.918227 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43956ef2-a54d-4814-a66b-f1e648162b37 admin admin] PATH_INFO: `/v3/projects/ad6940a9c1494bb08ebeee095ab169e3` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:19:09.920288 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43956ef2-a54d-4814-a66b-f1e648162b37 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ad6940a9c1494bb08ebeee095ab169e3)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:19:09.922216 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43956ef2-a54d-4814-a66b-f1e648162b37 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:19:09.971384 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-43956ef2-a54d-4814-a66b-f1e648162b37 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ad6940a9c1494bb08ebeee095ab169e3', 'request_id': 'req-43956ef2-a54d-4814-a66b-f1e648162b37'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:19:09.980237 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-43956ef2-a54d-4814-a66b-f1e648162b37 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ad6940a9c1494bb08ebeee095ab169e3', 'request_id': 'req-43956ef2-a54d-4814-a66b-f1e648162b37'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:19:09.986297 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-43956ef2-a54d-4814-a66b-f1e648162b37 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ad6940a9c1494bb08ebeee095ab169e3', 'request_id': 'req-43956ef2-a54d-4814-a66b-f1e648162b37'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:19:09.998144 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-43956ef2-a54d-4814-a66b-f1e648162b37 admin admin] The token cache is being invalidate because project ad6940a9c1494bb08ebeee095ab169e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:19:09.998474 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-43956ef2-a54d-4814-a66b-f1e648162b37 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:19:09.999202 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-43956ef2-a54d-4814-a66b-f1e648162b37 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:19:10.000756 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1437/2862] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:19:09 2022] DELETE /identity/v3/projects/ad6940a9c1494bb08ebeee095ab169e3 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:19:10.005825 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfda473-b84c-47d9-9ef7-de27a30d11b9 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:19:10.006147 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfda473-b84c-47d9-9ef7-de27a30d11b9 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:19:10.006528 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edfda473-b84c-47d9-9ef7-de27a30d11b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:19:10.024728 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-edfda473-b84c-47d9-9ef7-de27a30d11b9 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:19:10.026080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-edfda473-b84c-47d9-9ef7-de27a30d11b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:19:10.095981 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1426/2863] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:19:10.104658 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70dbaf-7428-4700-8481-4fa74815359d None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:19:10.104954 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70dbaf-7428-4700-8481-4fa74815359d None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:19:10.105234 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c70dbaf-7428-4700-8481-4fa74815359d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:19:10.124710 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9c70dbaf-7428-4700-8481-4fa74815359d None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:19:10.125941 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9c70dbaf-7428-4700-8481-4fa74815359d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:19:10.133969 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1438/2864] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:19:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:19:10.138582 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfda473-b84c-47d9-9ef7-de27a30d11b9 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:19:10.148434 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edfda473-b84c-47d9-9ef7-de27a30d11b9 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:19:10.153166 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4452f-d995-4070-8c3a-18f6fd066777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:19:10.153809 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4452f-d995-4070-8c3a-18f6fd066777 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:19:10.154097 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4452f-d995-4070-8c3a-18f6fd066777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:19:10.154414 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad4452f-d995-4070-8c3a-18f6fd066777 admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:19:10.155265 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad4452f-d995-4070-8c3a-18f6fd066777 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:19:10.155456 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad4452f-d995-4070-8c3a-18f6fd066777 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:19:10.157081 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad4452f-d995-4070-8c3a-18f6fd066777 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:19:10.170078 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1427/2865] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:19:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:19:10.175024 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70dbaf-7428-4700-8481-4fa74815359d None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:19:10.185365 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c70dbaf-7428-4700-8481-4fa74815359d None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:19:10.190189 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb4634-c205-4df8-b709-3aa7ac9b1504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:19:10.190878 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb4634-c205-4df8-b709-3aa7ac9b1504 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:19:10.191165 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb4634-c205-4df8-b709-3aa7ac9b1504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:19:10.191447 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cb4634-c205-4df8-b709-3aa7ac9b1504 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:19:10.192141 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb4634-c205-4df8-b709-3aa7ac9b1504 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:19:10.194084 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cb4634-c205-4df8-b709-3aa7ac9b1504 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:19:10.226049 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1439/2866] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:19:10 2022] POST /identity/v3/projects => generated 365 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:19:10.227175 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4452f-d995-4070-8c3a-18f6fd066777 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:19:10.232449 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:19:10.238678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad4452f-d995-4070-8c3a-18f6fd066777 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:19:10.300835 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3c044-a782-450d-b390-0e13fa5c61b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:19:10.301600 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3c044-a782-450d-b390-0e13fa5c61b8 admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:19:10.301882 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3c044-a782-450d-b390-0e13fa5c61b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:19:10.302161 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9c3c044-a782-450d-b390-0e13fa5c61b8 admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:19:10.302932 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3c044-a782-450d-b390-0e13fa5c61b8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:19:10.305421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9c3c044-a782-450d-b390-0e13fa5c61b8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:19:10.329007 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1428/2867] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:19:10 2022] POST /identity/v3/users => generated 283 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:19:10.330344 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb4634-c205-4df8-b709-3aa7ac9b1504 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:19:10.341326 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cb4634-c205-4df8-b709-3aa7ac9b1504 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:19:10.345969 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a330890-2ec0-4a2f-9951-4a1ab446a5fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:19:10.346651 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a330890-2ec0-4a2f-9951-4a1ab446a5fe admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:19:10.346939 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a330890-2ec0-4a2f-9951-4a1ab446a5fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:19:10.347222 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a330890-2ec0-4a2f-9951-4a1ab446a5fe admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:19:10.347950 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a330890-2ec0-4a2f-9951-4a1ab446a5fe admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:19:10.349690 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a330890-2ec0-4a2f-9951-4a1ab446a5fe admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:19:10.360935 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1440/2868] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:19:10.365258 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3c044-a782-450d-b390-0e13fa5c61b8 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:19:10.376978 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9c3c044-a782-450d-b390-0e13fa5c61b8 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:19:10.382271 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eea6714e-e908-4f01-93a1-da538eb74ca8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:19:10.382972 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6714e-e908-4f01-93a1-da538eb74ca8 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:19:10.383700 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6714e-e908-4f01-93a1-da538eb74ca8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:19:10.383700 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eea6714e-e908-4f01-93a1-da538eb74ca8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:19:10.384369 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6714e-e908-4f01-93a1-da538eb74ca8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:19:10.386060 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eea6714e-e908-4f01-93a1-da538eb74ca8 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:19:10.398401 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1429/2869] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:19:10 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:19:10.402712 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a330890-2ec0-4a2f-9951-4a1ab446a5fe admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:19:10.413382 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a330890-2ec0-4a2f-9951-4a1ab446a5fe admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:19:10.418357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d1a8b-083b-4593-92a7-88936b2ce64a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:19:10.419239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d1a8b-083b-4593-92a7-88936b2ce64a admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:19:10.419521 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d1a8b-083b-4593-92a7-88936b2ce64a admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:19:10.419796 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c1d1a8b-083b-4593-92a7-88936b2ce64a admin admin] PATH_INFO: `/v3/projects/83318466d07245798628bf7821d2d25f/users/299c06096a334739aacb0ee8f851769a/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:19:10.442837 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d1a8b-083b-4593-92a7-88936b2ce64a admin admin] RBAC: Authorizing `identity:create_grant(project_id=83318466d07245798628bf7821d2d25f, user_id=299c06096a334739aacb0ee8f851769a, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:19:10.445745 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c1d1a8b-083b-4593-92a7-88936b2ce64a admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:19:10.458676 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1441/2870] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:19:10 2022] PUT /identity/v3/projects/83318466d07245798628bf7821d2d25f/users/299c06096a334739aacb0ee8f851769a/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:19:10.470726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508151a1-ffc9-41b2-88aa-e61dee0354f8 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:19:10.470726 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508151a1-ffc9-41b2-88aa-e61dee0354f8 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:19:10.470933 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-508151a1-ffc9-41b2-88aa-e61dee0354f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:19:10.500875 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-508151a1-ffc9-41b2-88aa-e61dee0354f8 None None] MFA Rules not processed for user `299c06096a334739aacb0ee8f851769a`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:19:10.502129 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-508151a1-ffc9-41b2-88aa-e61dee0354f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:19:10.571518 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1430/2871] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:19:10 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:19:10.583021 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d1a8b-083b-4593-92a7-88936b2ce64a admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:19:10.585591 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-0c1d1a8b-083b-4593-92a7-88936b2ce64a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:19:10.607741 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:19:10.645301 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c1d1a8b-083b-4593-92a7-88936b2ce64a admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:19:10.649561 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf4a60-9ac6-4157-9826-582924b71345 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:19:10.650138 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf4a60-9ac6-4157-9826-582924b71345 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:19:10.650477 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf4a60-9ac6-4157-9826-582924b71345 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:19:10.650735 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2cf4a60-9ac6-4157-9826-582924b71345 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:19:10.658932 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf4a60-9ac6-4157-9826-582924b71345 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:19:10.660343 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2cf4a60-9ac6-4157-9826-582924b71345 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:19:10.669785 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1442/2872] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:19:10 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:19:20.568410 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=65,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:19:20.665757 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=17 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:19:26.347668 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249ff4e-1a11-4cbc-b83a-bc74d4df1ca5 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:19:26.348158 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249ff4e-1a11-4cbc-b83a-bc74d4df1ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:19:26.348259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3249ff4e-1a11-4cbc-b83a-bc74d4df1ca5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:19:26.366257 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-3249ff4e-1a11-4cbc-b83a-bc74d4df1ca5 None None] MFA Rules not processed for user `3c85d19cf42f4688bbc89a439d96ca69`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:19:26.367497 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3249ff4e-1a11-4cbc-b83a-bc74d4df1ca5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:19:26.422585 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1431/2873] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:19:26 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 76 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:19:36.417412 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=25 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:20:07.097863 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf4a60-9ac6-4157-9826-582924b71345 service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:20:07.099778 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-d2cf4a60-9ac6-4157-9826-582924b71345 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:20:07.175394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2cf4a60-9ac6-4157-9826-582924b71345 service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:20:07.181231 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06295e8-4ed5-4733-90a5-d5a6cc86d665 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee29b7f1cea64d6781e1df5c8021363f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:20:07.181899 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06295e8-4ed5-4733-90a5-d5a6cc86d665 service placement] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:07.182188 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06295e8-4ed5-4733-90a5-d5a6cc86d665 service placement] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:07.182508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06295e8-4ed5-4733-90a5-d5a6cc86d665 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:07.184308 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e06295e8-4ed5-4733-90a5-d5a6cc86d665 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:20:07.198531 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06295e8-4ed5-4733-90a5-d5a6cc86d665 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:20:07.200136 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e06295e8-4ed5-4733-90a5-d5a6cc86d665 service placement] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:20:07.212071 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1443/2874] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:20:07 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:20:13.306195 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249ff4e-1a11-4cbc-b83a-bc74d4df1ca5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:20:13.368419 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3249ff4e-1a11-4cbc-b83a-bc74d4df1ca5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:20:13.372907 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab71ac2-5a72-465f-b22a-820baf0d581c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:20:13.373773 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab71ac2-5a72-465f-b22a-820baf0d581c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:13.374062 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab71ac2-5a72-465f-b22a-820baf0d581c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:13.374369 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab71ac2-5a72-465f-b22a-820baf0d581c admin admin] PATH_INFO: `/v3/users/299c06096a334739aacb0ee8f851769a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:13.375559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab71ac2-5a72-465f-b22a-820baf0d581c admin admin] RBAC: Authorizing `identity:delete_user(user_id=299c06096a334739aacb0ee8f851769a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:20:13.377480 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab71ac2-5a72-465f-b22a-820baf0d581c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:20:13.423340 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8ab71ac2-5a72-465f-b22a-820baf0d581c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '299c06096a334739aacb0ee8f851769a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:13.423991 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8ab71ac2-5a72-465f-b22a-820baf0d581c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '299c06096a334739aacb0ee8f851769a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:13.431791 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8ab71ac2-5a72-465f-b22a-820baf0d581c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '299c06096a334739aacb0ee8f851769a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:13.432459 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8ab71ac2-5a72-465f-b22a-820baf0d581c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '299c06096a334739aacb0ee8f851769a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:13.444880 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-8ab71ac2-5a72-465f-b22a-820baf0d581c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '299c06096a334739aacb0ee8f851769a'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:13.456779 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1432/2875] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:20:13 2022] DELETE /identity/v3/users/299c06096a334739aacb0ee8f851769a => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:20:13.461494 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156566e-809b-435f-a5d2-33a684ec3c28 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:13.462088 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156566e-809b-435f-a5d2-33a684ec3c28 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:13.462088 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e156566e-809b-435f-a5d2-33a684ec3c28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:13.480045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-e156566e-809b-435f-a5d2-33a684ec3c28 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:20:13.481256 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e156566e-809b-435f-a5d2-33a684ec3c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:20:13.540331 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1444/2876] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:20:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:20:13.550585 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab71ac2-5a72-465f-b22a-820baf0d581c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:20:13.552574 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-8ab71ac2-5a72-465f-b22a-820baf0d581c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:20:13.611638 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab71ac2-5a72-465f-b22a-820baf0d581c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:20:13.615795 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd8ce6ae-2b6f-4b93-b529-8288be92e545 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:20:13.616434 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ce6ae-2b6f-4b93-b529-8288be92e545 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:13.616720 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ce6ae-2b6f-4b93-b529-8288be92e545 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:13.616995 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd8ce6ae-2b6f-4b93-b529-8288be92e545 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:13.625938 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ce6ae-2b6f-4b93-b529-8288be92e545 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:20:13.627317 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd8ce6ae-2b6f-4b93-b529-8288be92e545 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:20:13.637292 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1433/2877] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:20:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:20:13.992865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156566e-809b-435f-a5d2-33a684ec3c28 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:20:13.994888 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e156566e-809b-435f-a5d2-33a684ec3c28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:20:14.007818 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e156566e-809b-435f-a5d2-33a684ec3c28 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:20:14.012663 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3890311-13e0-42b2-892c-d367a209795e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:20:14.013451 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3890311-13e0-42b2-892c-d367a209795e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:14.013738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3890311-13e0-42b2-892c-d367a209795e admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:14.014023 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3890311-13e0-42b2-892c-d367a209795e admin admin] PATH_INFO: `/v3/projects/83318466d07245798628bf7821d2d25f` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:14.016078 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3890311-13e0-42b2-892c-d367a209795e admin admin] RBAC: Authorizing `identity:delete_project(project_id=83318466d07245798628bf7821d2d25f)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:20:14.018233 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3890311-13e0-42b2-892c-d367a209795e admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:20:14.069008 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f3890311-13e0-42b2-892c-d367a209795e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '83318466d07245798628bf7821d2d25f', 'request_id': 'req-f3890311-13e0-42b2-892c-d367a209795e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:14.080028 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f3890311-13e0-42b2-892c-d367a209795e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '83318466d07245798628bf7821d2d25f', 'request_id': 'req-f3890311-13e0-42b2-892c-d367a209795e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:14.086265 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f3890311-13e0-42b2-892c-d367a209795e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '83318466d07245798628bf7821d2d25f', 'request_id': 'req-f3890311-13e0-42b2-892c-d367a209795e'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:14.097803 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f3890311-13e0-42b2-892c-d367a209795e admin admin] The token cache is being invalidate because project 83318466d07245798628bf7821d2d25f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:20:14.098683 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f3890311-13e0-42b2-892c-d367a209795e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:14.099283 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-f3890311-13e0-42b2-892c-d367a209795e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:14.101166 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1445/2878] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:20:13 2022] DELETE /identity/v3/projects/83318466d07245798628bf7821d2d25f => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:20:14.106079 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca618c3c-a121-4e89-bf41-9a97486645b7 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:14.106421 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca618c3c-a121-4e89-bf41-9a97486645b7 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:14.106709 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca618c3c-a121-4e89-bf41-9a97486645b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:14.125003 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-ca618c3c-a121-4e89-bf41-9a97486645b7 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:20:14.126223 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ca618c3c-a121-4e89-bf41-9a97486645b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:20:14.143106 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:20:14.187482 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1434/2879] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:20:14.195725 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296555f-2bba-47d2-b545-fa2c83fa1fd3 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:14.196016 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296555f-2bba-47d2-b545-fa2c83fa1fd3 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:14.196299 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a296555f-2bba-47d2-b545-fa2c83fa1fd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:14.216825 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-a296555f-2bba-47d2-b545-fa2c83fa1fd3 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:20:14.218057 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a296555f-2bba-47d2-b545-fa2c83fa1fd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:20:14.227699 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1446/2880] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:20:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 11 14:20:14.231923 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca618c3c-a121-4e89-bf41-9a97486645b7 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:20:14.241619 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca618c3c-a121-4e89-bf41-9a97486645b7 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:20:14.246126 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0642d2f-d5c7-455a-9aac-62b64550ee5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:20:14.246809 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0642d2f-d5c7-455a-9aac-62b64550ee5c admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:14.247154 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0642d2f-d5c7-455a-9aac-62b64550ee5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:14.247360 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0642d2f-d5c7-455a-9aac-62b64550ee5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:14.248077 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0642d2f-d5c7-455a-9aac-62b64550ee5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80248) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 11 14:20:14.248383 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0642d2f-d5c7-455a-9aac-62b64550ee5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:20:14.249956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0642d2f-d5c7-455a-9aac-62b64550ee5c admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:20:14.261430 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1435/2881] 10.222.0.49 () {58 vars in 1152 bytes} [Mon Jul 11 14:20:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 11 14:20:14.265605 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296555f-2bba-47d2-b545-fa2c83fa1fd3 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:20:14.275705 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a296555f-2bba-47d2-b545-fa2c83fa1fd3 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:20:14.280450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36300339-2c82-4cd3-b4b5-604002f5f458 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:20:14.281113 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36300339-2c82-4cd3-b4b5-604002f5f458 admin admin] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:14.281394 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36300339-2c82-4cd3-b4b5-604002f5f458 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:14.281668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36300339-2c82-4cd3-b4b5-604002f5f458 admin admin] PATH_INFO: `/v3/projects` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:14.282391 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36300339-2c82-4cd3-b4b5-604002f5f458 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:20:14.284284 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36300339-2c82-4cd3-b4b5-604002f5f458 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:20:14.288209 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:20:14.313402 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1447/2882] 10.222.0.49 () {60 vars in 1152 bytes} [Mon Jul 11 14:20:14 2022] POST /identity/v3/projects => generated 399 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:20:14.317689 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0642d2f-d5c7-455a-9aac-62b64550ee5c admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:20:14.328153 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0642d2f-d5c7-455a-9aac-62b64550ee5c admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:20:14.383122 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172c0e6-31e8-4dd8-9582-09d8b53e5d6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:20:14.383785 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172c0e6-31e8-4dd8-9582-09d8b53e5d6e admin admin] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:14.384068 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172c0e6-31e8-4dd8-9582-09d8b53e5d6e admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:14.384343 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4172c0e6-31e8-4dd8-9582-09d8b53e5d6e admin admin] PATH_INFO: `/v3/users` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:14.385031 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172c0e6-31e8-4dd8-9582-09d8b53e5d6e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:20:14.386968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4172c0e6-31e8-4dd8-9582-09d8b53e5d6e admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:20:14.407679 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1436/2883] 10.222.0.49 () {60 vars in 1143 bytes} [Mon Jul 11 14:20:14 2022] POST /identity/v3/users => generated 300 bytes in 90 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 11 14:20:14.412357 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36300339-2c82-4cd3-b4b5-604002f5f458 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:20:14.422853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36300339-2c82-4cd3-b4b5-604002f5f458 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:20:14.427779 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b81a0d-9e84-4f9a-a14f-fe2cfa1db421 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:20:14.428437 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b81a0d-9e84-4f9a-a14f-fe2cfa1db421 admin admin] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:14.428724 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b81a0d-9e84-4f9a-a14f-fe2cfa1db421 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:14.429001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b81a0d-9e84-4f9a-a14f-fe2cfa1db421 admin admin] PATH_INFO: `/v3/roles` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:14.429731 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b81a0d-9e84-4f9a-a14f-fe2cfa1db421 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:20:14.431366 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b81a0d-9e84-4f9a-a14f-fe2cfa1db421 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:20:14.441422 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1448/2884] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:20:14.445851 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172c0e6-31e8-4dd8-9582-09d8b53e5d6e admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:20:14.455749 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4172c0e6-31e8-4dd8-9582-09d8b53e5d6e admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:20:14.461117 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a977e42-d7e9-41de-a380-7f464070a3b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:20:14.461771 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a977e42-d7e9-41de-a380-7f464070a3b1 admin admin] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:14.462062 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a977e42-d7e9-41de-a380-7f464070a3b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:14.462379 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a977e42-d7e9-41de-a380-7f464070a3b1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:14.463116 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a977e42-d7e9-41de-a380-7f464070a3b1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:20:14.464703 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a977e42-d7e9-41de-a380-7f464070a3b1 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:20:14.474678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1437/2885] 10.222.0.49 () {58 vars in 1121 bytes} [Mon Jul 11 14:20:14 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 11 14:20:14.478796 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b81a0d-9e84-4f9a-a14f-fe2cfa1db421 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:20:14.489229 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b81a0d-9e84-4f9a-a14f-fe2cfa1db421 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:20:14.493901 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed217bb9-50ba-4054-b3c3-6bac342479b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:20:14.494831 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed217bb9-50ba-4054-b3c3-6bac342479b2 admin admin] REQUEST_METHOD: `PUT` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:14.495111 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed217bb9-50ba-4054-b3c3-6bac342479b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:14.495410 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed217bb9-50ba-4054-b3c3-6bac342479b2 admin admin] PATH_INFO: `/v3/projects/b68bc3f8f19a4d9f8b866a685bff8f03/users/15b014139e9c4715b2ab761aea84fdbb/roles/827f1f1239754bf797e64f270ba1a80c` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:14.510238 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed217bb9-50ba-4054-b3c3-6bac342479b2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b68bc3f8f19a4d9f8b866a685bff8f03, user_id=15b014139e9c4715b2ab761aea84fdbb, role_id=827f1f1239754bf797e64f270ba1a80c)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:20:14.512896 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed217bb9-50ba-4054-b3c3-6bac342479b2 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:20:14.525695 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1449/2886] 10.222.0.49 () {60 vars in 1482 bytes} [Mon Jul 11 14:20:14 2022] PUT /identity/v3/projects/b68bc3f8f19a4d9f8b866a685bff8f03/users/15b014139e9c4715b2ab761aea84fdbb/roles/827f1f1239754bf797e64f270ba1a80c => generated 0 bytes in 47 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:20:14.536770 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9fc87-064d-449f-8777-70f46d4972d5 None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:14.537069 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9fc87-064d-449f-8777-70f46d4972d5 None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:14.537353 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fc9fc87-064d-449f-8777-70f46d4972d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:14.564239 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-1fc9fc87-064d-449f-8777-70f46d4972d5 None None] MFA Rules not processed for user `15b014139e9c4715b2ab761aea84fdbb`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:20:14.565461 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-1fc9fc87-064d-449f-8777-70f46d4972d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:20:14.628934 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1438/2887] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:20:14 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:20:14.640738 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed217bb9-50ba-4054-b3c3-6bac342479b2 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:20:14.642464 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ed217bb9-50ba-4054-b3c3-6bac342479b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:20:14.699910 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed217bb9-50ba-4054-b3c3-6bac342479b2 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:20:14.703552 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b68615a-4745-4854-96bc-96358cad5d65 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:20:14.704142 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68615a-4745-4854-96bc-96358cad5d65 service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:14.704422 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68615a-4745-4854-96bc-96358cad5d65 service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:14.704700 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b68615a-4745-4854-96bc-96358cad5d65 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:14.714207 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68615a-4745-4854-96bc-96358cad5d65 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:20:14.715687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b68615a-4745-4854-96bc-96358cad5d65 service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:20:14.725007 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1450/2888] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:20:14 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:20:24.626013 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:INSERT=3 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:20:24.720717 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=47,keystone:INSERT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:20:26.943786 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9fc87-064d-449f-8777-70f46d4972d5 None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:20:27.008276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fc9fc87-064d-449f-8777-70f46d4972d5 None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:20:27.013312 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44172858-503f-45c4-bf61-db053d462773 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:20:27.014383 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44172858-503f-45c4-bf61-db053d462773 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:27.014678 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44172858-503f-45c4-bf61-db053d462773 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:27.014956 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44172858-503f-45c4-bf61-db053d462773 admin admin] PATH_INFO: `/v3/users/15b014139e9c4715b2ab761aea84fdbb` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:27.016201 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44172858-503f-45c4-bf61-db053d462773 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15b014139e9c4715b2ab761aea84fdbb)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:20:27.018170 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44172858-503f-45c4-bf61-db053d462773 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:20:27.080088 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-44172858-503f-45c4-bf61-db053d462773 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15b014139e9c4715b2ab761aea84fdbb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:27.080975 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-44172858-503f-45c4-bf61-db053d462773 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15b014139e9c4715b2ab761aea84fdbb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:27.089416 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-44172858-503f-45c4-bf61-db053d462773 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15b014139e9c4715b2ab761aea84fdbb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:27.090144 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-44172858-503f-45c4-bf61-db053d462773 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15b014139e9c4715b2ab761aea84fdbb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:27.108381 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-44172858-503f-45c4-bf61-db053d462773 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15b014139e9c4715b2ab761aea84fdbb'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:27.121844 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1439/2889] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:20:26 2022] DELETE /identity/v3/users/15b014139e9c4715b2ab761aea84fdbb => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:20:27.126425 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd2a96-2c8d-43e0-855f-118e5b1f08a9 None None] REQUEST_METHOD: `POST` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:27.126906 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd2a96-2c8d-43e0-855f-118e5b1f08a9 None None] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:27.127031 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fcd2a96-2c8d-43e0-855f-118e5b1f08a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:27.146061 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.auth.core [None req-9fcd2a96-2c8d-43e0-855f-118e5b1f08a9 None None] MFA Rules not processed for user `e549d830b8bf4a6d8e16e2dad340366b`. Rule list: `[]` (Enabled: `True`). {{(pid=80247) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:20:27.147380 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9fcd2a96-2c8d-43e0-855f-118e5b1f08a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:20:27.206476 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1451/2890] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:20:27.216701 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44172858-503f-45c4-bf61-db053d462773 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:20:27.218125 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-44172858-503f-45c4-bf61-db053d462773 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:20:27.274607 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44172858-503f-45c4-bf61-db053d462773 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:20:27.278424 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d6e3ef2-fad7-4843-a6ee-8e353f92de21 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:20:27.279080 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e3ef2-fad7-4843-a6ee-8e353f92de21 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:27.279361 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e3ef2-fad7-4843-a6ee-8e353f92de21 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:27.279636 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6e3ef2-fad7-4843-a6ee-8e353f92de21 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:27.289243 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e3ef2-fad7-4843-a6ee-8e353f92de21 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:20:27.290697 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d6e3ef2-fad7-4843-a6ee-8e353f92de21 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:20:27.300292 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1440/2891] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:20:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:20:27.567426 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd2a96-2c8d-43e0-855f-118e5b1f08a9 None None] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:20:27.569045 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-9fcd2a96-2c8d-43e0-855f-118e5b1f08a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:20:27.583193 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fcd2a96-2c8d-43e0-855f-118e5b1f08a9 None None] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:20:27.589657 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382afe7d-d242-45c7-a281-45edb6cf7bb5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:20:27.590620 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382afe7d-d242-45c7-a281-45edb6cf7bb5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:27.590914 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382afe7d-d242-45c7-a281-45edb6cf7bb5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:27.591190 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-382afe7d-d242-45c7-a281-45edb6cf7bb5 admin admin] PATH_INFO: `/v3/projects/b68bc3f8f19a4d9f8b866a685bff8f03` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:27.592442 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382afe7d-d242-45c7-a281-45edb6cf7bb5 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b68bc3f8f19a4d9f8b866a685bff8f03)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:20:27.594508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-382afe7d-d242-45c7-a281-45edb6cf7bb5 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:20:27.643310 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-382afe7d-d242-45c7-a281-45edb6cf7bb5 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b68bc3f8f19a4d9f8b866a685bff8f03', 'request_id': 'req-382afe7d-d242-45c7-a281-45edb6cf7bb5'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:27.650485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-382afe7d-d242-45c7-a281-45edb6cf7bb5 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b68bc3f8f19a4d9f8b866a685bff8f03', 'request_id': 'req-382afe7d-d242-45c7-a281-45edb6cf7bb5'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:27.656180 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-382afe7d-d242-45c7-a281-45edb6cf7bb5 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b68bc3f8f19a4d9f8b866a685bff8f03', 'request_id': 'req-382afe7d-d242-45c7-a281-45edb6cf7bb5'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:27.667472 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-382afe7d-d242-45c7-a281-45edb6cf7bb5 admin admin] The token cache is being invalidate because project b68bc3f8f19a4d9f8b866a685bff8f03 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80247) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:20:27.667678 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-382afe7d-d242-45c7-a281-45edb6cf7bb5 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:27.668318 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-382afe7d-d242-45c7-a281-45edb6cf7bb5 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:20:27.669916 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1452/2892] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:20:27 2022] DELETE /identity/v3/projects/b68bc3f8f19a4d9f8b866a685bff8f03 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:20:31.215070 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d271577-4fc5-4af8-9356-c94fe3e666de None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:31.215566 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d271577-4fc5-4af8-9356-c94fe3e666de None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:31.215681 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d271577-4fc5-4af8-9356-c94fe3e666de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:31.233333 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0d271577-4fc5-4af8-9356-c94fe3e666de None None] MFA Rules not processed for user `72a0be934143483e8cdcc25fb7b3c915`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:20:31.234567 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0d271577-4fc5-4af8-9356-c94fe3e666de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:20:31.251707 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:20:31.292475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1441/2893] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:20:31 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:20:31.524364 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382afe7d-d242-45c7-a281-45edb6cf7bb5 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:20:31.525800 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-382afe7d-d242-45c7-a281-45edb6cf7bb5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:20:31.584070 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-382afe7d-d242-45c7-a281-45edb6cf7bb5 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:20:31.587698 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84befb39-0f55-487b-835b-264b08da0c08 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:20:31.588350 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84befb39-0f55-487b-835b-264b08da0c08 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:20:31.588668 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84befb39-0f55-487b-835b-264b08da0c08 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:20:31.588947 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84befb39-0f55-487b-835b-264b08da0c08 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:20:31.590771 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-84befb39-0f55-487b-835b-264b08da0c08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:20:31.601293 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84befb39-0f55-487b-835b-264b08da0c08 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:20:31.602749 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84befb39-0f55-487b-835b-264b08da0c08 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:20:31.610967 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1453/2894] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:20:31 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:20:41.285215 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=15 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:20:41.607318 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:21:35.719021 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d271577-4fc5-4af8-9356-c94fe3e666de None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:21:35.720451 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0d271577-4fc5-4af8-9356-c94fe3e666de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:21:35.774700 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84befb39-0f55-487b-835b-264b08da0c08 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:21:35.776270 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-84befb39-0f55-487b-835b-264b08da0c08 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:21:35.798955 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d271577-4fc5-4af8-9356-c94fe3e666de None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:21:35.805086 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34fca1-3d92-4bdf-a5d1-7796d64386de service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee29b7f1cea64d6781e1df5c8021363f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:21:35.805833 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34fca1-3d92-4bdf-a5d1-7796d64386de service placement] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:21:35.806180 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34fca1-3d92-4bdf-a5d1-7796d64386de service placement] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:21:35.806547 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac34fca1-3d92-4bdf-a5d1-7796d64386de service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:21:35.808319 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ac34fca1-3d92-4bdf-a5d1-7796d64386de service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:21:35.820701 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34fca1-3d92-4bdf-a5d1-7796d64386de service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:21:35.822269 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac34fca1-3d92-4bdf-a5d1-7796d64386de service placement] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:21:35.832927 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1442/2895] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:21:35 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 11 14:21:35.851501 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84befb39-0f55-487b-835b-264b08da0c08 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:21:35.856254 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452543e6-1998-49ac-94a7-cd50d90c87fe admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:21:35.857160 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452543e6-1998-49ac-94a7-cd50d90c87fe admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:21:35.857450 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452543e6-1998-49ac-94a7-cd50d90c87fe admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:21:35.857730 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452543e6-1998-49ac-94a7-cd50d90c87fe admin admin] PATH_INFO: `/v3/users/72a0be934143483e8cdcc25fb7b3c915` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:21:35.858962 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452543e6-1998-49ac-94a7-cd50d90c87fe admin admin] RBAC: Authorizing `identity:delete_user(user_id=72a0be934143483e8cdcc25fb7b3c915)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:21:35.861011 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-452543e6-1998-49ac-94a7-cd50d90c87fe admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:21:35.911565 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-452543e6-1998-49ac-94a7-cd50d90c87fe admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72a0be934143483e8cdcc25fb7b3c915'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:21:35.924088 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-452543e6-1998-49ac-94a7-cd50d90c87fe admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72a0be934143483e8cdcc25fb7b3c915'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:21:35.932710 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-452543e6-1998-49ac-94a7-cd50d90c87fe admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72a0be934143483e8cdcc25fb7b3c915'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:21:35.933503 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-452543e6-1998-49ac-94a7-cd50d90c87fe admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72a0be934143483e8cdcc25fb7b3c915'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:21:35.940594 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-452543e6-1998-49ac-94a7-cd50d90c87fe admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72a0be934143483e8cdcc25fb7b3c915'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:21:35.947105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1454/2896] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:21:35 2022] DELETE /identity/v3/users/72a0be934143483e8cdcc25fb7b3c915 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:21:36.120731 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34fca1-3d92-4bdf-a5d1-7796d64386de service placement] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:21:36.133765 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac34fca1-3d92-4bdf-a5d1-7796d64386de service placement] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:21:36.203111 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a06c672-a434-487c-b38b-3dbe36f44924 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:21:36.203996 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06c672-a434-487c-b38b-3dbe36f44924 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:21:36.204276 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06c672-a434-487c-b38b-3dbe36f44924 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:21:36.204551 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a06c672-a434-487c-b38b-3dbe36f44924 admin admin] PATH_INFO: `/v3/projects/58346265e4c244dcb15cd6ec2817773a` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:21:36.205789 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a06c672-a434-487c-b38b-3dbe36f44924 admin admin] RBAC: Authorizing `identity:delete_project(project_id=58346265e4c244dcb15cd6ec2817773a)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:21:36.207822 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a06c672-a434-487c-b38b-3dbe36f44924 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:21:36.254549 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3a06c672-a434-487c-b38b-3dbe36f44924 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '58346265e4c244dcb15cd6ec2817773a', 'request_id': 'req-3a06c672-a434-487c-b38b-3dbe36f44924'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:21:36.259935 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3a06c672-a434-487c-b38b-3dbe36f44924 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '58346265e4c244dcb15cd6ec2817773a', 'request_id': 'req-3a06c672-a434-487c-b38b-3dbe36f44924'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:21:36.267211 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3a06c672-a434-487c-b38b-3dbe36f44924 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '58346265e4c244dcb15cd6ec2817773a', 'request_id': 'req-3a06c672-a434-487c-b38b-3dbe36f44924'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:21:36.278716 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3a06c672-a434-487c-b38b-3dbe36f44924 admin admin] The token cache is being invalidate because project 58346265e4c244dcb15cd6ec2817773a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:21:36.279006 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3a06c672-a434-487c-b38b-3dbe36f44924 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:21:36.279643 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-3a06c672-a434-487c-b38b-3dbe36f44924 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:21:36.281184 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1443/2897] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:21:36 2022] DELETE /identity/v3/projects/58346265e4c244dcb15cd6ec2817773a => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:21:45.943615 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:21:46.275580 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=83,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:22:06.852887 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452543e6-1998-49ac-94a7-cd50d90c87fe admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:22:06.854508 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-452543e6-1998-49ac-94a7-cd50d90c87fe admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:22:06.914467 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-452543e6-1998-49ac-94a7-cd50d90c87fe admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:22:06.965971 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fe042-3981-4e98-b552-9877d3612f5b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:22:06.966675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fe042-3981-4e98-b552-9877d3612f5b service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:22:06.966965 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fe042-3981-4e98-b552-9877d3612f5b service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:22:06.967252 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69fe042-3981-4e98-b552-9877d3612f5b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:22:06.969085 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a69fe042-3981-4e98-b552-9877d3612f5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:22:07.036135 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69fe042-3981-4e98-b552-9877d3612f5b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:22:07.037580 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69fe042-3981-4e98-b552-9877d3612f5b service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:22:07.048173 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1455/2898] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:22:06 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 196 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:22:16.113338 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907516c-219f-4783-8d17-9495f085710c None None] REQUEST_METHOD: `POST` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:22:16.113923 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907516c-219f-4783-8d17-9495f085710c None None] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:22:16.113923 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0907516c-219f-4783-8d17-9495f085710c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:22:16.132256 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.auth.core [None req-0907516c-219f-4783-8d17-9495f085710c None None] MFA Rules not processed for user `3c85d19cf42f4688bbc89a439d96ca69`. Rule list: `[]` (Enabled: `True`). {{(pid=80248) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 11 14:22:16.133506 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0907516c-219f-4783-8d17-9495f085710c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:22:16.141183 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1444/2899] 10.222.0.49 () {58 vars in 957 bytes} [Mon Jul 11 14:22:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 11 14:22:16.183540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fe042-3981-4e98-b552-9877d3612f5b service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:22:16.184986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-a69fe042-3981-4e98-b552-9877d3612f5b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:22:16.195947 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69fe042-3981-4e98-b552-9877d3612f5b service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:22:16.200342 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62ac3c-8a66-4d29-9a20-37fec87efaba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '6bb1616f632d4c67828cce3fc41945b7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:22:16.200998 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62ac3c-8a66-4d29-9a20-37fec87efaba service cinder] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:22:16.201284 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62ac3c-8a66-4d29-9a20-37fec87efaba service cinder] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:22:16.201567 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c62ac3c-8a66-4d29-9a20-37fec87efaba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:22:16.203388 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-1c62ac3c-8a66-4d29-9a20-37fec87efaba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:22:16.274599 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=100 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:22:16.277733 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c62ac3c-8a66-4d29-9a20-37fec87efaba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:22:16.279567 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c62ac3c-8a66-4d29-9a20-37fec87efaba service cinder] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:22:16.291866 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1456/2900] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:22:16 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:22:17.015964 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907516c-219f-4783-8d17-9495f085710c None None] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:22:17.017391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0907516c-219f-4783-8d17-9495f085710c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:22:17.087143 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0907516c-219f-4783-8d17-9495f085710c None None] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:22:17.090836 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32f813-c0fb-406b-852b-ab3c182b3543 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:22:17.091475 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32f813-c0fb-406b-852b-ab3c182b3543 service neutron] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:22:17.091814 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32f813-c0fb-406b-852b-ab3c182b3543 service neutron] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:22:17.092017 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc32f813-c0fb-406b-852b-ab3c182b3543 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:22:17.093882 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dc32f813-c0fb-406b-852b-ab3c182b3543 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:22:17.108241 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32f813-c0fb-406b-852b-ab3c182b3543 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:22:17.109647 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc32f813-c0fb-406b-852b-ab3c182b3543 service neutron] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:22:17.121441 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1445/2901] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:22:17 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:22:26.286648 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=2 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:22:27.115820 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=29 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:22:34.993777 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62ac3c-8a66-4d29-9a20-37fec87efaba service cinder] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:22:35.003592 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c62ac3c-8a66-4d29-9a20-37fec87efaba service cinder] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:22:35.007305 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab75ba-075b-4152-90a4-e3eeac5e2577 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:22:35.007957 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab75ba-075b-4152-90a4-e3eeac5e2577 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:22:35.008248 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab75ba-075b-4152-90a4-e3eeac5e2577 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:22:35.009063 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ab75ba-075b-4152-90a4-e3eeac5e2577 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:22:35.010938 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e4ab75ba-075b-4152-90a4-e3eeac5e2577 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:22:35.077907 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ab75ba-075b-4152-90a4-e3eeac5e2577 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:22:35.078751 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4ab75ba-075b-4152-90a4-e3eeac5e2577 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:22:35.089871 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1457/2902] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:22:34 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:22:37.083155 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32f813-c0fb-406b-852b-ab3c182b3543 service neutron] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:22:37.084749 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-dc32f813-c0fb-406b-852b-ab3c182b3543 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:22:37.100016 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc32f813-c0fb-406b-852b-ab3c182b3543 service neutron] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:22:37.107559 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7dab6-c351-437f-a392-93544d2b431b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:22:37.108311 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7dab6-c351-437f-a392-93544d2b431b service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:22:37.108679 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7dab6-c351-437f-a392-93544d2b431b service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:22:37.108961 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ede7dab6-c351-437f-a392-93544d2b431b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:22:37.111391 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-ede7dab6-c351-437f-a392-93544d2b431b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:22:37.127283 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7dab6-c351-437f-a392-93544d2b431b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:22:37.128870 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ede7dab6-c351-437f-a392-93544d2b431b service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:22:37.139756 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1446/2903] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:22:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:22:45.084349 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=30 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:22:47.135152 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=6 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:23:06.120997 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab75ba-075b-4152-90a4-e3eeac5e2577 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:23:06.122574 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-e4ab75ba-075b-4152-90a4-e3eeac5e2577 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:23:06.134492 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ab75ba-075b-4152-90a4-e3eeac5e2577 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:23:06.138747 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6a60e-3068-4196-a78d-eee98a57d709 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:23:06.139423 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6a60e-3068-4196-a78d-eee98a57d709 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:23:06.139708 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6a60e-3068-4196-a78d-eee98a57d709 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:23:06.139986 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd6a60e-3068-4196-a78d-eee98a57d709 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:23:06.141795 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-7cd6a60e-3068-4196-a78d-eee98a57d709 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:23:06.152545 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6a60e-3068-4196-a78d-eee98a57d709 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:23:06.154053 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd6a60e-3068-4196-a78d-eee98a57d709 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:23:06.166272 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1458/2904] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:23:06 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:23:16.159687 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=6 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:24:27.016995 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7dab6-c351-437f-a392-93544d2b431b service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:24:27.030868 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ede7dab6-c351-437f-a392-93544d2b431b service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:24:27.037254 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef6521-b5af-45a2-ac7a-12b46ee7d42a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ef5dac3f311e43809a46995bab56e780', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['member', 'service', 'ResellerAdmin', 'admin', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:24:27.037995 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef6521-b5af-45a2-ac7a-12b46ee7d42a service nova] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:24:27.038283 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef6521-b5af-45a2-ac7a-12b46ee7d42a service nova] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:24:27.038615 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bef6521-b5af-45a2-ac7a-12b46ee7d42a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:24:27.040460 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3bef6521-b5af-45a2-ac7a-12b46ee7d42a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:24:27.059344 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef6521-b5af-45a2-ac7a-12b46ee7d42a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:24:27.060945 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bef6521-b5af-45a2-ac7a-12b46ee7d42a service nova] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:24:27.075624 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1447/2905] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:24:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:24:37.068884 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=6 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:24:39.523162 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6a60e-3068-4196-a78d-eee98a57d709 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:24:39.532604 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd6a60e-3068-4196-a78d-eee98a57d709 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:24:39.536865 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38fe67-af75-4bf7-afff-5f9d5523d8e9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ce1c85ee44ef45d2b631e51112b5ea0a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:24:39.537993 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38fe67-af75-4bf7-afff-5f9d5523d8e9 service neutron] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:24:39.538316 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38fe67-af75-4bf7-afff-5f9d5523d8e9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:24:39.538603 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac38fe67-af75-4bf7-afff-5f9d5523d8e9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:24:39.541675 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ac38fe67-af75-4bf7-afff-5f9d5523d8e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:24:39.624460 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38fe67-af75-4bf7-afff-5f9d5523d8e9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:24:39.625807 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac38fe67-af75-4bf7-afff-5f9d5523d8e9 service neutron] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:24:39.637853 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1459/2906] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:24:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:24:46.079047 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef6521-b5af-45a2-ac7a-12b46ee7d42a service nova] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:24:46.080389 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-3bef6521-b5af-45a2-ac7a-12b46ee7d42a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:24:46.139885 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38fe67-af75-4bf7-afff-5f9d5523d8e9 service neutron] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:24:46.141431 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-ac38fe67-af75-4bf7-afff-5f9d5523d8e9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:24:46.156108 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac38fe67-af75-4bf7-afff-5f9d5523d8e9 service neutron] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:24:46.156525 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bef6521-b5af-45a2-ac7a-12b46ee7d42a service nova] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:24:46.161581 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e812a-01a7-4f74-8778-652849e1be1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:24:46.162443 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e812a-01a7-4f74-8778-652849e1be1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:24:46.162580 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141fe11-a40e-4fc3-b6f1-46e4d7ae59f2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee29b7f1cea64d6781e1df5c8021363f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:24:46.162693 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e812a-01a7-4f74-8778-652849e1be1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:24:46.162968 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949e812a-01a7-4f74-8778-652849e1be1b admin admin] PATH_INFO: `/v3/users/3c85d19cf42f4688bbc89a439d96ca69` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:24:46.163316 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141fe11-a40e-4fc3-b6f1-46e4d7ae59f2 service placement] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:24:46.163610 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141fe11-a40e-4fc3-b6f1-46e4d7ae59f2 service placement] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:24:46.163832 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f141fe11-a40e-4fc3-b6f1-46e4d7ae59f2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:24:46.164336 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e812a-01a7-4f74-8778-652849e1be1b admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c85d19cf42f4688bbc89a439d96ca69)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:24:46.165794 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-f141fe11-a40e-4fc3-b6f1-46e4d7ae59f2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:24:46.168241 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949e812a-01a7-4f74-8778-652849e1be1b admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:24:46.180139 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f141fe11-a40e-4fc3-b6f1-46e4d7ae59f2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:24:46.181741 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f141fe11-a40e-4fc3-b6f1-46e4d7ae59f2 service placement] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:24:46.192290 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1448/2907] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:24:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:24:46.220130 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-949e812a-01a7-4f74-8778-652849e1be1b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c85d19cf42f4688bbc89a439d96ca69'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:46.233239 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-949e812a-01a7-4f74-8778-652849e1be1b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c85d19cf42f4688bbc89a439d96ca69'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:46.240742 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-949e812a-01a7-4f74-8778-652849e1be1b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c85d19cf42f4688bbc89a439d96ca69'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:46.241319 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-949e812a-01a7-4f74-8778-652849e1be1b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c85d19cf42f4688bbc89a439d96ca69'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:46.248143 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-949e812a-01a7-4f74-8778-652849e1be1b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c85d19cf42f4688bbc89a439d96ca69'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:46.253962 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1460/2908] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:24:46 2022] DELETE /identity/v3/users/3c85d19cf42f4688bbc89a439d96ca69 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 11 14:24:46.426473 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141fe11-a40e-4fc3-b6f1-46e4d7ae59f2 service placement] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:24:46.438968 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f141fe11-a40e-4fc3-b6f1-46e4d7ae59f2 service placement] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:24:46.498452 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1141be-c87d-48b1-a262-50331368de2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:24:46.499322 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1141be-c87d-48b1-a262-50331368de2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:24:46.499613 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1141be-c87d-48b1-a262-50331368de2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:24:46.499893 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1141be-c87d-48b1-a262-50331368de2f admin admin] PATH_INFO: `/v3/projects/67d8660c31bf48e9a2c6a5edb2060bde` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:24:46.501083 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1141be-c87d-48b1-a262-50331368de2f admin admin] RBAC: Authorizing `identity:delete_project(project_id=67d8660c31bf48e9a2c6a5edb2060bde)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:24:46.503027 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1141be-c87d-48b1-a262-50331368de2f admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:24:46.549548 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0a1141be-c87d-48b1-a262-50331368de2f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '67d8660c31bf48e9a2c6a5edb2060bde', 'request_id': 'req-0a1141be-c87d-48b1-a262-50331368de2f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:46.555208 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0a1141be-c87d-48b1-a262-50331368de2f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '67d8660c31bf48e9a2c6a5edb2060bde', 'request_id': 'req-0a1141be-c87d-48b1-a262-50331368de2f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:46.563196 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0a1141be-c87d-48b1-a262-50331368de2f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '67d8660c31bf48e9a2c6a5edb2060bde', 'request_id': 'req-0a1141be-c87d-48b1-a262-50331368de2f'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:46.575987 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0a1141be-c87d-48b1-a262-50331368de2f admin admin] The token cache is being invalidate because project 67d8660c31bf48e9a2c6a5edb2060bde was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:24:46.576280 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0a1141be-c87d-48b1-a262-50331368de2f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:46.576988 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-0a1141be-c87d-48b1-a262-50331368de2f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:46.578602 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1449/2909] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:24:46 2022] DELETE /identity/v3/projects/67d8660c31bf48e9a2c6a5edb2060bde => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:24:46.582540 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e812a-01a7-4f74-8778-652849e1be1b admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:24:46.584001 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-949e812a-01a7-4f74-8778-652849e1be1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:24:46.597869 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949e812a-01a7-4f74-8778-652849e1be1b admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:24:46.655586 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048814d7-0c30-4253-b14e-31022faea3f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:24:46.656471 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048814d7-0c30-4253-b14e-31022faea3f4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:24:46.656800 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048814d7-0c30-4253-b14e-31022faea3f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:24:46.657068 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048814d7-0c30-4253-b14e-31022faea3f4 admin admin] PATH_INFO: `/v3/users/febf83e79bcc4f6f90e2c7a2aaf23c80` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:24:46.658259 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048814d7-0c30-4253-b14e-31022faea3f4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=febf83e79bcc4f6f90e2c7a2aaf23c80)` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:24:46.660269 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048814d7-0c30-4253-b14e-31022faea3f4 admin admin] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:24:46.666184 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:24:46.710080 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-048814d7-0c30-4253-b14e-31022faea3f4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'febf83e79bcc4f6f90e2c7a2aaf23c80'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:46.722336 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-048814d7-0c30-4253-b14e-31022faea3f4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'febf83e79bcc4f6f90e2c7a2aaf23c80'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:46.729852 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-048814d7-0c30-4253-b14e-31022faea3f4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'febf83e79bcc4f6f90e2c7a2aaf23c80'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:46.730485 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-048814d7-0c30-4253-b14e-31022faea3f4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'febf83e79bcc4f6f90e2c7a2aaf23c80'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:46.737553 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.notifications [None req-048814d7-0c30-4253-b14e-31022faea3f4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'febf83e79bcc4f6f90e2c7a2aaf23c80'} {{(pid=80247) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:46.743479 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1461/2910] 10.222.0.49 () {58 vars in 1223 bytes} [Mon Jul 11 14:24:46 2022] DELETE /identity/v3/users/febf83e79bcc4f6f90e2c7a2aaf23c80 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:24:46.922418 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1141be-c87d-48b1-a262-50331368de2f admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:24:46.924160 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-0a1141be-c87d-48b1-a262-50331368de2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:24:46.937827 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1141be-c87d-48b1-a262-50331368de2f admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:24:46.958435 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:24:47.001005 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344d3fb-0aa7-420e-ac12-eb633d37ca23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e549d830b8bf4a6d8e16e2dad340366b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '5898aef39ca74b7d8b99cb1aa700e9c2', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:24:47.001913 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344d3fb-0aa7-420e-ac12-eb633d37ca23 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:24:47.002198 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344d3fb-0aa7-420e-ac12-eb633d37ca23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:24:47.002500 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2344d3fb-0aa7-420e-ac12-eb633d37ca23 admin admin] PATH_INFO: `/v3/projects/f867bcd9ba0445bcbac83aaf3693ee6c` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:24:47.003701 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2344d3fb-0aa7-420e-ac12-eb633d37ca23 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f867bcd9ba0445bcbac83aaf3693ee6c)` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:24:47.005610 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2344d3fb-0aa7-420e-ac12-eb633d37ca23 admin admin] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:24:47.052182 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2344d3fb-0aa7-420e-ac12-eb633d37ca23 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f867bcd9ba0445bcbac83aaf3693ee6c', 'request_id': 'req-2344d3fb-0aa7-420e-ac12-eb633d37ca23'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:47.057915 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2344d3fb-0aa7-420e-ac12-eb633d37ca23 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f867bcd9ba0445bcbac83aaf3693ee6c', 'request_id': 'req-2344d3fb-0aa7-420e-ac12-eb633d37ca23'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:47.065922 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2344d3fb-0aa7-420e-ac12-eb633d37ca23 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f867bcd9ba0445bcbac83aaf3693ee6c', 'request_id': 'req-2344d3fb-0aa7-420e-ac12-eb633d37ca23'} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:47.078597 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2344d3fb-0aa7-420e-ac12-eb633d37ca23 admin admin] The token cache is being invalidate because project f867bcd9ba0445bcbac83aaf3693ee6c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80248) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 11 14:24:47.078889 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2344d3fb-0aa7-420e-ac12-eb633d37ca23 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:47.079578 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.notifications [None req-2344d3fb-0aa7-420e-ac12-eb633d37ca23 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80248) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 11 14:24:47.081231 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1450/2911] 10.222.0.49 () {58 vars in 1232 bytes} [Mon Jul 11 14:24:46 2022] DELETE /identity/v3/projects/f867bcd9ba0445bcbac83aaf3693ee6c => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 11 14:24:56.740033 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:24:57.074822 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG dbcounter [-] [80248] Writing DB stats keystone:SELECT=42,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80248) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:25:13.659589 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048814d7-0c30-4253-b14e-31022faea3f4 admin admin] Authenticating user token {{(pid=80247) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:25:13.660958 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-048814d7-0c30-4253-b14e-31022faea3f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:25:13.796955 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048814d7-0c30-4253-b14e-31022faea3f4 admin admin] Validating token access rules against request {{(pid=80247) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:25:13.854105 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac52068-ced7-48f1-9c4d-c827d2258c22 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee29b7f1cea64d6781e1df5c8021363f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80247) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:25:13.854784 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac52068-ced7-48f1-9c4d-c827d2258c22 service placement] REQUEST_METHOD: `GET` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:25:13.855073 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac52068-ced7-48f1-9c4d-c827d2258c22 service placement] SCRIPT_NAME: `/identity` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:25:13.855345 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac52068-ced7-48f1-9c4d-c827d2258c22 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80247) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:25:13.857214 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.fernet_utils [None req-2ac52068-ced7-48f1-9c4d-c827d2258c22 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80247) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:25:13.869046 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac52068-ced7-48f1-9c4d-c827d2258c22 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:25:13.870683 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac52068-ced7-48f1-9c4d-c827d2258c22 service placement] RBAC: Authorization granted {{(pid=80247) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:25:13.881563 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: [pid: 80247|app: 0|req: 1462/2912] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:25:13 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 11 14:25:23.877099 dv-cinder-848429-3-iscsi devstack@keystone.service[80247]: DEBUG dbcounter [-] [80247] Writing DB stats keystone:SELECT=89 {{(pid=80247) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 11 14:30:22.434051 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344d3fb-0aa7-420e-ac12-eb633d37ca23 admin admin] Authenticating user token {{(pid=80248) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 11 14:30:22.435711 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.fernet_utils [None req-2344d3fb-0aa7-420e-ac12-eb633d37ca23 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80248) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 11 14:30:22.477625 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2344d3fb-0aa7-420e-ac12-eb633d37ca23 admin admin] Validating token access rules against request {{(pid=80248) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 11 14:30:22.483033 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a67abc4-78a0-4c18-8e8c-c350681ab6bb service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ee29b7f1cea64d6781e1df5c8021363f', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '08ee65f8b338491f80e7beac5e316b23', 'project_domain_id': 'default', 'roles': ['admin', 'reader', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80248) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 11 14:30:22.483691 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67abc4-78a0-4c18-8e8c-c350681ab6bb service placement] REQUEST_METHOD: `GET` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 11 14:30:22.483980 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67abc4-78a0-4c18-8e8c-c350681ab6bb service placement] SCRIPT_NAME: `/identity` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 11 14:30:22.484259 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a67abc4-78a0-4c18-8e8c-c350681ab6bb service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80248) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 11 14:30:22.493495 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67abc4-78a0-4c18-8e8c-c350681ab6bb service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 11 14:30:22.495143 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a67abc4-78a0-4c18-8e8c-c350681ab6bb service placement] RBAC: Authorization granted {{(pid=80248) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 11 14:30:22.505741 dv-cinder-848429-3-iscsi devstack@keystone.service[80248]: [pid: 80248|app: 0|req: 1451/2913] 10.222.0.49 () {60 vars in 1365 bytes} [Mon Jul 11 14:30:22 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 72 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0)